Agent ROI: the break-even math against a human baseline

1. The break-even inequality

Cost per solved task is not a business case on its own. It is one side of an inequality whose other side is what the work costs without the agent. The calculator implements exactly this, and writing it out makes the assumptions visible:

human cost per task   = (minutes / 60) * loaded cost per hour
agent attempt cost    = (tokens_per_task / 1e6) * blended $/1M
agent solved cost     = attempt cost * (1 + retry_overhead) / success_rate
saving per task       = human cost per task - agent solved cost
break-even volume     = fixed monthly cost / saving per task

Three properties of that chain deserve attention before any number goes in a deck. The human baseline is a management input — loaded rate and minutes per task — not a published price, and it usually dominates the result. The success rate divides: failures are paid for, so a cheaper attempt that fails more often can lose to an expensive one that resolves. And the last line is a volume question, not an efficiency question: the agent breaks even on its fixed cost at a specific number of solved tasks per month.

2. Cost per solved task, not cost per task

The ordering changes once you divide by the resolved rate. These are the SWE-bench Verified rows from the benchmark snapshot (2026-10-27), with the solved cost computed from the published per-task cost:

The reordering is the lesson. AutoCodeRover looks mid-priced at $0.70 per task; at a 46.2% resolved rate it is the second most expensive per solved task in the set. GitHub Copilot moves ahead of Claude Code on solved cost while looking cheaper per attempt. A purchasing spreadsheet sorted on cost per task would rank these agents differently from one sorted on what the agent actually delivers, and only the second ordering survives contact with a budget.

3. Three human baselines, three verdicts

Take the Claude Code row — 72.8% resolved at $0.85, so $1.17 per solved task — and compare it against three baselines for the same task. The tiers are ours, chosen to represent a small routine change, a routine engineering ticket and a specialist judgement call; substitute your own from the calculator, whose default is 45 minutes at a $65/hour loaded rate, or $48.75 per task.

The last column is the one worth defending. It is the total human time, error correction and review you can afford per task before the saving disappears, and five minutes of review at the calculator's $65/hour default already costs $5.42 — more than four times the token cost of the task itself.

Read it as a threshold. At a $5 baseline, any review longer than about three and a half minutes wipes out the agent's advantage. At $15 the same five-minute review still leaves $8.42 per task. At $40 it is noise. Push the baseline lower and the window closes: against a $2 task the agent saves $0.83 at zero review time, and two minutes of review moves the break-even baseline to $3.34, above the task it is meant to replace. This is why the same agent can be an obvious win in one team and a distraction in another without either team being wrong.

4. What the token bill leaves out

The calculator states its own exclusions — sandbox and VM time, evaluation harnesses, human review, latency — and the list is the right place to start a business case rather than to finish one.

• Review time is the largest omission and it scales with trust, not with tokens. The first months after adoption are the most expensive ones.
• Rework after a wrong answer is a probability-weighted cost. At a $40 baseline, if 1 in 10 delivered outputs needs a human to redo it, expected cost per delivered task is $1.17 + $4.00 = $5.17; at 1 in 5 it is $9.17. The rework rate is the number to instrument, and no vendor publishes it.
• Adoption cost — integration, a deterministic evaluation set, prompt and tool maintenance, on-call attention — is a fixed monthly number in the calculator's terms, and it is what turns the break-even into a volume requirement.
• Price drift. The pricing snapshot is dated 2026-10-27 and list-price only. A model change, a cache policy change or a negotiated rate moves the agent side of the inequality without moving the human side.

5. Why benchmark numbers do not transfer

The temptation is to take $1.17 per solved task and multiply by ticket volume. Four reasons that overstates the case:

• "Solved" means a test suite passes, not that a reviewer accepts it. SWE-bench rewards patches that turn a failing test green. Production rewards changes that are correct, reviewable and maintainable.
• The 24.1M-token row has no published success rate. The dataset leaves Claude Code's success rate empty because the Artificial Analysis index publishes benchmark scores (Index 59, FrontierCode 48%), not a task success rate — so that row cannot be turned into a cost per solved task at all. The 72.8% figure belongs to a different, narrower task set.
• The task mix is different. Curated, self-contained issues understate the ambiguity, context hunting and flaky infrastructure of a normal backlog, all of which raise tokens and lower success simultaneously.
• Much of the dataset has no per-task cost at all. In the 2026-10-27 snapshot, 20 of 30 projects publish no cost per task and 27 of 30 publish no token count — a reminder that for most production agents the bill is measured by the operator, not quoted by the vendor.

The useful role for a benchmark row is to bound expectations, not to be the forecast. The project cards and leaderboards tell you which agents are in which cost class; the cost per solved task definition tells you how the division works; and your own telemetry is the only source for your own success rate.

6. Break-even volume is the real decision

With a fixed monthly cost, the saving per task becomes a payback requirement. Using $2,000 a month as an illustrative adoption cost — integration and evaluation, not tokens — the same three baselines need quite different volumes: 522 solved tasks per month at a $5 baseline, 145 at $15, and 52 at $40.

That is the whole argument in one table. At a $5 baseline, the agent has to replace a routine task roughly 500 times a month before its own overhead is covered; at $40, under two a day is enough. Volume, not price per token, decides whether the deployment is rational, and volume is the number teams most often estimate from enthusiasm. The honest sequence is to measure a frozen task set, compute a defensible solved cost, add the review and rework terms you can defend, and only then divide the fixed cost. Do it in the calculator — and read its exclusion list before you quote the output.

Two conclusions survive every assumption above. First, the break-even line is dominated by the human baseline and the review budget, not by the model's list price; the token bill is the smallest term in the inequality for most task types. Second, the benchmark figures are evidence about the mechanism and the ordering, not a forecast of your bill. Both are reasons to measure rather than to argue.

Sources

• SWE-bench Verified resolved rates and per-task costs — benchmarks.json, snapshot 2026-10-27; row values cross-checked against the TrustTheBench agents leaderboard because the canonical leaderboard page is client-rendered.
• Claude Code per-task tokens, cost and the note that no task success rate is published — projects.json (claude-code), snapshot 2026-10-27; source figures from the Artificial Analysis Coding Agent Index v2.0 comparison.
• Solved-cost formula, metric definitions and provenance tiers — AgentBurn methodology.
• Human baseline defaults (45 minutes, $65/hour), break-even definition and the exclusion list — AgentBurn cost & ROI calculator.
• Model list prices — models.json, snapshot 2026-10-27.