DeepSeek V4 Pro 0813 与 GPT-5.6 Sol 在 DeepSWE 上的成本与编码对比
Together AI 在 DeepSWE 全部 113 个任务上各跑 4 次对比 DeepSeek V4 Pro 0813 与 GPT-5.6 Sol：Sol 单次通过率 72.7% 领先 Pro 的 62.8%，但 Pro 每次 rollout 仅 0.24 美元，约为 Sol 8.37 美元的 35 分之一，pass@4 上 Pro 以 88.5% 反超 Sol 的 85.8%。先用 Pro、测试失败再升级到 Sol 的级联方案可解决 83.0% 的任务，单任务成本 3.35 美元，高于 Sol 单独使用的 72.7%，也高于理想单次路由的 80.8%。Sol 更快更简洁（中位 53 步、17 分钟、59k 输出 token），但失败时 20% 会破坏仓库原有测试套件，Pro 这一比例为 11%。
推荐理由
AI 评分 71
原文用 904 次 rollout 对比两款模型在 DeepSWE 上的成本与通过率，并给出可复用的级联路由方案。
主题
正文
当前显示原文，中文全文译文尚未就绪。
Key Takeaways
Don't pick one. Run DeepSeek V4 Pro 0813 first, escalate to GPT-5.6 Sol when the tests fail. That cascade solves 83.0% of DeepSWE tasks at \$3.35 each. Sol alone solves 72.7% at \$8.37. Ten points better, 60% cheaper.
In our DeepSeek V4 Pro 0813 vs GPT-5.6 Sol comparison on DeepSWE, a benchmark that tests a model's software engineering ability across many task types and programming languages, GPT-5.6 Sol is the best single-shot engineer on the board, and DeepSeek V4 Pro 0813 costs one-thirty-fifth as much. The interesting question is not which is more accurate on the first try (Sol, clearly) but what that 35x price gap actually buys, and whether the lower-cost model's ceiling closes it.
DeepSWE · Head to Head
We ran DeepSeek V4 Pro 0813 (max) against GPT-5.6 Sol (max) on all 113 DeepSWE tasks, four trials each, from the published per-trial records: 904 rollouts in total (452 each). Sol is the precision flagship; Pro is the value outlier. Every figure below comes from this run, so it can differ from other public DeepSeek V4 Pro 0813 vs GPT-5.6 Sol scorecards.
Single shot, Sol leads clearly: 72.7% pass@1 to Pro's 62.8% (official scoring). But the gap narrows with every retry and then inverts: at two attempts Pro is already close (78.5 vs 81.0), and at four Pro's 88.5% pass@4 passes Sol's 85.8%. The lower-cost model has the wider reach; it needs more than one attempt, and its attempts are nearly free. If your workload lets you run best-of-k, the accuracy argument for Sol largely disappears.
At \$0.24 a rollout, DeepSeek V4 Pro 0813 is 35x cheaper than Sol (\$8.37), which in value terms is 260 solves per \$100 against Sol's 9. What the lower price does not buy is speed or brevity: Pro takes a median 146 steps and 35 minutes and emits 101k output tokens, versus Sol's tight 53 steps, 17 minutes, and 59k tokens. Sol is the fast, concise specialist; Pro reaches similar coverage the long way around. If a human is waiting, Sol earns its premium on latency alone; if a budget or a queue is waiting, nothing here is close to Pro.
Decompose pass@1 into coverage and reliability and the split is clean. Sol is the precision corner: 84.5% reliability and 61 tasks solved four-for-four, the marks of a model that nails what it touches. DeepSeek V4 Pro 0813 trades to the other axis: wider coverage (88.5% vs 85.8%) but far lower reliability (71.0%) and fewer solid tasks (35 vs 61). That is the pass@4 crossover seen from the side: Pro touches more of the benchmark than Sol but converts each touch less often per shot.
The failure profiles differ sharply. Sol breaks the repository's existing test suite in 20% of its failures, the GPT-family regression signature. DeepSeek V4 Pro 0813 is far more conservative at 11%; when it fails, it usually fails with a near miss and the baseline intact. So the cheaper model is also the safer one to accept unreviewed: put a full regression gate around Sol before you take its diff, and Pro needs that guardrail less.
Sol's quality edge is broad: it wins 6 of 8 domains, led by data modeling and serialization at 92% (28 points over Pro), plus query/config (80), concurrency (72), build/ops (73), program analysis (64), and protocol conformance (59). The two it does not take are narrow: language and runtime internals is a 75-75 tie, and DeepSeek V4 Pro 0813 edges stateful reactivity (66 vs 64), its one domain win. Everywhere the task demands nailing an exact serialization contract, Sol pulls away.
Sol sweeps four of five (Python 74, Go 79, TypeScript 66, JavaScript 75), and the margins are wide on Python and Go. The exception is Rust, where DeepSeek V4 Pro 0813 edges ahead 65 to 60: the one language Sol underperforms its class and the one Pro wins outright. If your stack is Python or Go, this is Sol's board by a comfortable margin; if it is Rust, Pro is fractionally better and vastly cheaper.
Moderately. Per-task correlation is 0.54, the closest of the DeepSeek-Pro pairings. They both solve 90 tasks; Pro alone gets 10, Sol alone gets 7, and 6 defeat both. Their union covers 107 of 113 (94.7%). One asymmetry matters for a portfolio: DeepSeek V4 Pro 0813 sweeps no task that Sol misses entirely, while Sol goes four-for-four on two tasks that Pro never solves (pebble-durability-wait-apis, textual-kitty-key-phases). So Pro does not add much new reach on top of Sol; what it adds is the ability to clear the shared tasks at minimal cost.
Which is exactly why the cascade wins. Run DeepSeek V4 Pro 0813 first and escalate to Sol only when your test suite rejects the answer: 83.0% solved at \$3.35 per task. That is ten points above Sol alone (72.7%) for less than half of Sol's own per-task price (\$8.37). The near-free first stage clears most of the queue, so Sol's premium price applies only to the hard remainder, and the tasks that reach Sol get a second independent attempt on top. The cascade even beats a perfect one-shot oracle router (80.8%), because two independent attempts beat one perfect pick. Accuracy is the same whichever model leads, but Pro-first is far cheaper (\$3.35 vs \$8.44), so always lead with the lower-cost model.
GPT-5.6 Sol is the single-model pick when correctness and latency both matter on the first try: highest pass@1, highest reliability, fastest and most concise, and six of eight domains. You pay 35x DeepSeek V4 Pro 0813 for that, and you must guardrail its 20% regression rate. DeepSeek V4 Pro 0813 is the value and best-of-k pick: near-flagship coverage, the higher four-shot ceiling, a cleaner failure profile, at \$0.24 a rollout. But the sharpest use of the pair is neither alone, it is Pro as Sol's front-end. A low-cost first stage that clears most of the work turns Sol into a cost you only pay on the tasks that need it, and buys flagship-beating coverage for under half of Sol's price.
DeepSWE · Full results
It depends on the metric. GPT-5.6 Sol wins single-attempt quality on DeepSWE (pass@1 72.7% vs 62.8%), solves more tasks four-for-four (61 vs 35), and is much faster per rollout. DeepSeek V4 Pro 0813 wins pass@4 (88.5% vs 85.8%) at 35x lower cost per rollout, so it is the stronger value pick for high-volume or retry-tolerant agent work.
In our run, DeepSeek V4 Pro 0813 cost \$0.24 per rollout versus \$8.37 for GPT-5.6 Sol at max effort, roughly 35x cheaper. Measured per solved task, Pro returned 260 solves per \$100 against Sol's 9, about 30x the solved work per dollar.
GPT-5.6 Sol leads Python (74), Go (79), TypeScript (66), and JavaScript (75), and wins 6 of 8 task domains, led by data modeling and serialization at 92%. DeepSeek V4 Pro 0813 takes Rust (65 vs 60) and edges stateful reactivity (66 vs 64). Sol is also the faster model per rollout; Pro reaches wider coverage across multiple attempts.
Yes, if you can verify results. Running DeepSeek V4 Pro 0813 first and escalating to GPT-5.6 Sol when your test suite rejects the output reaches 83.0% at \$3.35 per task, beating Sol alone (72.7% at \$8.37) and a perfect one-shot oracle router (80.8%). Together the two cover 107 of 113 tasks.
pass@k measures whether at least one of k attempts at a task passes the hidden test suite. pass@1 rewards getting it right first try; higher k rewards a model that can eventually reach a solution across several tries. DeepSeek V4 Pro 0813's edge grows as k increases.