Cost per task is the number to compare
A coding agent doesn’t answer once. It reads files, runs commands, checks results and tries again, sending its growing context back to the model at every step. The price per token tells you what each of those tokens costs; it says nothing about how many the model will use. Cost per task, the average spend for one complete attempt, captures both.
On DeepSWE the two rankings barely agree. Ranked by price per token, Claude Sonnet 5 is one of the cheapest models here. Ranked by cost per task, it is the most expensive, by a factor of two. GPT-6 Astra shows the reverse: the highest per-token rate, yet one of the lowest costs per task. Sonnet 5 at max effort averaged 72.6M tokens per attempt; GPT-6 Astra averaged 1.5M.
| Model (effort) | Pass rate | $ / 1M tokens | $ / task | Tokens / task |
|---|---|---|---|---|
| Claude Sonnet 5 (max) | 53.8% | $0.36 | $26.40 | 72.6M |
| Claude Fable 5 (xhigh) | 69.9% | $1.79 | $13.41 | 7.4M |
| Claude Opus 5 (max) | 73.6% | $0.78 | $11.84 | 15.1M |
| Kimi K3 (max), open-weight | 68.5% | $0.48 | $4.65 | 9.8M |
| GPT-6 Astra (xhigh) | 74.1% | $2.98 | $4.43 | 1.5M |
| GLM-5.3 (max), open-weight | 69.0% | $0.30 | $3.99 | 13.4M |
| Gemini 3.8 Flash (high) | 73.8% | $0.11 | $2.36 | 21.9M |
| GPT-6.1 Sol (high), reported | 75.2% | ~$0.40 | $0.65 | — |
A strong result doesn’t need the most expensive run. Gemini 3.8 Flash passed 73.8% of attempts for $2.36 each, within half a point of the best measured pass rate (GPT-6 Astra’s 74.1%) at about half its cost per task and a fifth of Claude Opus 5’s.
Caching sets the effective price
Because an agent resends most of its context at every step, the bulk of its tokens are repeats. Providers bill a repeated prompt prefix as a cached read, at a fraction of the normal input price: GPT-6 Astra charges $1 per 1M cached tokens against $10 for uncached input, and GPT-6.1 Sol lists $0.10 against $2. On DeepSWE, cached reads were 89% of GPT-6 Astra’s tokens and 96–99% for every other model we measured.
That has three practical consequences:
- The blended per-token rate is mostly the cache-read price. Sonnet 5’s low $0.36 rate is low because 99% of its tokens were cheap cached reads. It was still the most expensive per task because it read so many of them.
- Longer runs cost more than proportionally. Each extra step rereads a longer context, so a run that takes twice as many steps usually costs more than twice as much.
- Cache misses are expensive. Changing early context mid-run (rewriting instructions, reordering tools) or letting the cache expire during a slow step sends the whole prefix back at full input price. Keep the start of the prompt stable and put changing content at the end.
Where open-weight models make sense
GLM-5.3 and Kimi K3 passed 69.0% and 68.5% of attempts, about 5 points behind the leaders, for $3.99 and $4.65 per task. That is 34–39% of Claude Opus 5’s cost per task. On hosted pricing alone they are not the cheapest option here: Gemini 3.8 Flash scored higher for less. Their case rests on control rather than list price:
- Code that can’t leave your infrastructure. Running the model yourself keeps source and prompts inside your environment, which matters for regulated or air-gapped work.
- High, steady volume. On your own hardware the marginal cost of a task falls as utilization rises, and price changes stay in your hands.
- Well-scoped stages in a larger workflow. An open-weight model can handle implementation stages that are checked automatically, while a frontier model plans and reviews.
- A pinned version. You decide when the model changes, and can fine-tune it on your own code.
If you self-host, plan for the same token volumes: 9.8M to 13.4M tokens per attempt here. Prefix caching in your serving stack matters as much as it does for hosted models. Check the license too: both models carry conditions for very large commercial users.
Method and caveats
- Data comes from Datacurve’s DeepSWE v1.1 trials: 113 tasks, 4 attempts each, run with the mini-swe-agent harness. Each model is shown at the reasoning effort where it scored best. Attempts that ended in provider or infrastructure errors are excluded; agent timeouts count as failures.
- Price per token is a blended rate: total spend divided by all input, cached and output tokens. Costs use the prices displayed on the DeepSWE site.
- GPT-6.1 Sol was not part of Datacurve’s runs. Its pass rate and cost per task are OpenAI-reported; its per-token rate is our estimate from list prices using GPT-5.6 Sol’s measured token mix.
- Four Claude Fable 5 attempts have no recorded cost and are left out of its average.