NaiveAI Promises 2,100 Tokens per Second. How Well Does It Code?

NaiveAI, a Chinese company, has released Naive-N0.5-Flash, an open-weights model aimed at AI coding work and tuned to maximize tokens per second. Latent Space benchmarked its ultrafast mode against DeepSeek V4.1 Flash 1017 and Z.ai's GLM 5.3 Flash, the two most popular models on OpenRouter as of the end of October 2026, and also asked all three to build the same web apps from identical prompts. The model is a further-trained evolution of Xiaomi's MiMo-V2.5, paired with NaiveAI's own NaiveRT inference engine. NaiveAI's long-term pitch is a model largely built by AI and trained to do AI research itself, with recursive self-improvement as the goal.
MiMo-V2.5 is a 310B-parameter sparse Mixture-of-Experts model with 15B active parameters, trained on 48T tokens. A sparse MoE activates only a fraction of its parameters per token, which keeps inference cheaper than a dense model of the same total size. Its language backbone follows the hybrid design of MiMo-V2-Flash, mixing sliding-window attention with global self-attention. NaiveAI started from those weights and, through continued multi-stage training, adapted the checkpoint to DeepSeek Sparse Attention layers, using roughly 3.25T additional tokens on top of the original 48T. About 3T of that was trained at a fixed 1M context length with a constant learning rate. This suggests NaiveAI prioritized long-context robustness, since training at full length rather than ramping up avoids the distribution shift that short-context pretraining leaves behind.
Published benchmarks from the official model cards and technical posts place Naive-N0.5-Flash as follows: 71.9 on NL2Repo-Bench (against 68.2 for DeepSeek V4.1 Flash 1017 and 56.3 for GLM-5.3-Flash), 67.8 on DeepSWE v1.1 (against 77.0 and 63.4), and 32.4 on Agent's Last Exam (against 35.4 and 26.3). Its coding ability therefore sits between the two comparison models overall, while it leads on NL2Repo-Bench. Artificial Analysis does not yet publish performance statistics for NaiveAI models; for context, the Intelligence Index scores cited for the comparisons are 45 for DeepSeek V4.1 Flash 1017 (Max) and 43 for GLM-5.3-Flash.
The speed tests used 10 randomly sampled text-only questions at each of three difficulty levels: MATH-500 for easy, GPQA-Diamond for medium and HLE-Diamond for hard, so 30 questions per model, with a 256K maximum completion-token limit. Throughput figures came from OpenRouter for the two comparison models but from server-side measurements for Naive-N0.5-Flash, so Latent Space declined to claim an exact speed multiple. Naive-N0.5-Flash approached or passed 1,000 tokens per second in both thinking configurations.
Verbosity swung sharply. On the hard HLE questions, Naive-N0.5-Flash with thinking on averaged about 108,500 completion tokens, roughly nine times the GLM 5.3 Flash (high) average and eleven times the DeepSeek V4.1 Flash 1017 (high) average. Medians differ from means: across all 30 questions the median completion for Naive-N0.5-Flash with thinking on was about 3,200 tokens against an average above 51,000, meaning a few requests generated extremely long reasoning traces and dragged the mean up. The same pattern, weaker, appeared in the other models. End-to-end time is the fairer speed measure: the median question took 3.1 seconds on Naive-N0.5-Flash (thinking), versus 5.3 seconds for GLM 5.3 Flash (high) and 10.1 seconds for DeepSeek V4.1 Flash 1017 (high). Means were closer, at 45.3 seconds, 45.0 seconds and 86.9 seconds respectively. On hard questions, Naive-N0.5-Flash (thinking) was slower on average than DeepSeek V4.1 Flash 1017 (high), 89.8 seconds against 79.7. The sample is small and answer accuracy was not graded.
In the app experiments, six model-and-effort combinations each received three prompts: an interactive 3D Earth and Solar System, a seeded 3D tree with tapering branches and thousands of leaves under wind animation, and a browser port of the book Ray Tracing in One Weekend. Naive-N0.5-Flash (thinking) produced the best solar system app, with sunspots and city lights glowing on Earth's night side; DeepSeek V4.1 Flash 1017 (high) came close but lacked an asteroid belt and showed less surface detail, while GLM 5.3 Flash (high) struggled with lighting and overly sensitive speed controls. On the tree, DeepSeek and Naive both did well, though Naive-N0.5-Flash with thinking off produced no usable app at all, and wind animations showed glitches in the better results, with GLM's tree wobbling like jelly at high wind. On the ray tracer, all three thinking configurations worked, but DeepSeek's was the most efficient and used the full screen width; Naive's was correct with less efficient JavaScript, and GLM's had the more basic interface.
Pricing targets are $0.10 per million input tokens, $0.40 per million output tokens and $0.01 per million cache-read tokens, placing it in the $0.20 to $0.60 per million output-token band shared with several capable Flash models. Final spend also depends on token consumption, which follows the thinking budget and question difficulty. Latent Space's takeaway is a peak near 2,000 tokens per second and nearly 1,000 sustained during thinking-heavy work, with median response times several times shorter than the comparisons.
The speed rests on speculative decoding and a custom engine. The reported decoding speeds were measured on a single eight-GPU Blackwell node using the Naive-N0.5-Flash-FP8-Draft drafter in NaiveRT, tuned for single-stream decode latency rather than batch throughput. Speculative decoding has a small draft model propose several tokens that the target model verifies in parallel; realized speed depends on how many drafts are accepted per round, which varies with checkpoint pair, prompt, context length, sampling settings and content. Acceptance was 70.6% with thinking disabled but only 43.4% with thinking enabled. Decode ran at about 984 tokens per second in the thinking phase and 1,816 in the answer phase, with roughly 2,100 over a single second treated as an upper bound rather than a typical rate. A full speculative round (draft, verify, sampling, commit) takes 3.4 ms in NaiveRT versus 12.3 ms in SGLang, a 72.4% reduction; the thinking-disabled campaign measured about 3.45 ms per round.
Why it matters: teams that iterate quickly through code generation may find the median-response advantage worth the token spend, though the caveats about server-side versus OpenRouter measurement and the small sample mean an exact ranking against DeepSeek and GLM remains unproven. The larger point, an inference this analysis draws from the acceptance-rate and engine numbers, is that fast autoregressive rollouts lower the cost of large-scale reinforcement learning loops, which is the stated premise behind NaiveAI's self-improvement ambitions. Developers should benchmark on their own prompts and grading before switching, since speed without accuracy is of limited value.
Based on reporting from the original publisher. Visit the source for full context and later updates.
Publisher excerpt
NaiveAI could potentially top Artificial Analysis’s speed rankings. Our hands-on test suggests its coding quality holds up against DeepSeek.