AivexaNewsSearch
AI news for builders and product teamsChecked every hour

DeepSeek-V4 Flash 0731 vs GPT-5.6 Luna on DeepSWE: Cost and Coding

Collected Oct 29, 2026

Together AI published a head-to-head evaluation of DeepSeek-V4 Flash 0731 against GPT-5.6 Luna on DeepSWE, running 900 rollouts across 113 tasks drawn from real, long-horizon feature requests in live open-source repositories. Each model got four trials per task, graded pass or fail by a hidden test suite; the per-trial records split 452 rollouts to DeepSeek and 448 to Luna. All figures come from this single run, and Together notes they can diverge from other public scorecards.

The headline result is a clean capability split. Under DeepSWE's official scoring, Luna leads pass@1 at 67.2% against DeepSeek's 53.3% — a 14-point gap, or roughly 26% more accurate in relative terms. Luna also holds the lead at every equal attempt count: 81.6% versus 70.1% at two attempts, and 90.3% versus 80.5% at four. On raw solving ability, the evaluation is unambiguous that this is not close.

The economics tell a different story. DeepSeek-V4 Flash is the cheapest model on the DeepSWE board at about $0.10 per rollout, against $0.61 for Luna — roughly six times the price. Measured per solved task, DeepSeek returns 532 solves per $100 to Luna's 110, a 4.8x value edge. Together highlights one subtlety: DeepSeek's pass@2 of 70.1% already edges Luna's single-shot 67.2%, so two cheap attempts cost about $0.20 combined and can match the flagship's first-try quality if something can pick the winning run. Pass@k, for readers new to the metric, measures whether at least one of k attempts passes the hidden tests, so it rewards models that eventually get there across retries rather than first-try precision.

The speed result inverts the usual "flash" narrative. DeepSeek was the slower of the two, taking a median 23 minutes and 148 steps against Luna's 16 minutes and 92, and emitting more output — 104k median tokens versus 70k. Together attributes part of the wall-clock gap to the model's newness, expecting it to narrow as inference engines get tuned for it. In practice, DeepSeek's advantage is money, not time, which matters for anyone running these workloads under latency budgets rather than batch conditions.

Failure modes give the cheap model a second, quieter win. When DeepSeek fails, it breaks the repository's existing test suite in only 9% of failures; Luna does so in 15%, which Together describes as the GPT-family regression signature seen at similar rates across related models. Both mostly fail by near miss (69% for DeepSeek, 66% for Luna). The practical reading is that teams deploying Luna should gate it behind a full regression run, while DeepSeek needs that guardrail less.

Breaking the 113 tasks down by domain, Luna wins seven of eight, with its largest margins in reasoning-heavy work: program analysis (69 versus 33), concurrency and durability (70 versus 38), and language and runtime internals (86 versus 59), each roughly a 30-point gap. DeepSeek's single domain win is query and config languages, 78 versus 70 — SQL builders, window functions, keyset pagination, config parsers. Together's framing is that structured, schema-shaped, convention-following work is where the cheap model is genuinely competitive, while tasks requiring a hard invariant held across a whole system are where capability separates.

Luna also wins all five programming languages in the breakdown, but the margins vary usefully. DeepSeek is respectable on Rust (55 versus 60) and Go (62 versus 79). JavaScript is its worst result anywhere in the comparison: 35 versus 60, a 25-point hole, with Python also soft at 49 versus 65. For a JavaScript-heavy stack, this suggests the cheap model is a false economy unless paired with escalation; for config, query or Rust work, the gap largely closes.

Similarity between the two models is lower than typical cheap-tier pairings. Per-task correlation is 0.50, and the overlap is lopsided: both solve 87 tasks, Luna alone solves 15 more, and DeepSeek alone adds only 4. No task is swept by DeepSeek that Luna misses entirely, while Luna sweeps four that DeepSeek never lands. Their union covers 106 of 113 tasks (93.8%), but almost all of that extra reach is Luna's — so as a pure diversity play, DeepSeek adds little.

That is what makes the cascade result interesting. Running DeepSeek first and escalating to Luna only when the test suite rejects the answer solves 78.9% of tasks at $0.385 each — more accurate than Luna alone by 11.7 points and about 37% cheaper, landing at roughly 63% of Luna's per-task cost. The mechanism is that DeepSeek clears about 53% of the queue for a dime apiece, so Luna's charge only touches the hard remainder, and escalated tasks get a second independent attempt. The cascade even beats a hypothetical perfect one-shot router (74.3%), because two swings beat one perfect pick. Accuracy is identical whichever model leads, but DeepSeek-first costs $0.385 versus $0.64, so leading cheap is strictly better.

The pair's ceiling is that 93.8% union; the seven tasks neither model ever solves would need a stronger third model rather than a smarter router. My reading is that this ceiling, not the cost figure, is the constraint that matters for anyone tempted to treat the cascade as a drop-in replacement for a frontier model on hard systems work.

Why it matters: teams running coding agents at volume now have a measured case for treating model selection as a routing problem rather than a single pick — a cheap first stage plus a flagship escalation beat the flagship on both accuracy and cost in this run, subject to verifier quality. The caveat is that the cascade depends on a reliable test suite to judge failure; without automated verification, escalation never triggers and the accuracy gain disappears. Teams on JavaScript-heavy codebases should treat DeepSeek as a front-end only, not a substitute.

Read at Together AI

Based on reporting from the original publisher. Visit the source for full context and later updates.

Publisher excerpt

We ran 900 DeepSWE rollouts on DeepSeek-V4 Flash and GPT-5.6 Luna. Luna leads pass@1 by 14 points; DeepSeek delivers 4.8x the solves per dollar.