Fireworks AI analyzes DeepSWE v1.1 model routing potential

Fireworks AI released an analysis on September 21, 2026, testing 18 models on 113 coding tasks in DeepSWE v1.1 to examine the potential of routing models by task. Its published results show that selecting the best model per task after the fact can reach a 97.6% pass rate at a cost of $1.88 per task, while the best-performing single model, GPT-6 Astra, has a pass rate of 74.1% at a cost of $6.52 per task. If limited to the open-weight model pool, routing can still reach 90.3% at $1.45 per task. The authors also note that these figures come from oracle routing selected after the fact, which represents an upper-bound perspective; a real router must make choices before seeing the results. Its FireRouter, across 2,334 production coding sessions over four weeks, reduced cost from $15.81 for the single model Opus 5 to $7.42.

AI-generated from public reports

1. Sep 21· 2 reports

   Fireworks AI analyzes DeepSWE model routing potential

   Source:Fireworks AI