Retry overhead: the 31% of an agent bill nobody budgets

1. What counts as retry

AgentBurn defines retry overhead on the methodology page as the share of spend caused by tool errors, loops and failed validation. The definition is narrow on purpose. Mapping a repository is work, not retry. Editing files is work. Running the test suite for the first time is work. Retry is the spend that exists only because an attempt failed to converge: the second patch, the third test run, the re-read of a 40,000-token traceback, the compaction that rebuilds state after the context window filled up.

Most budgets never split the bill this way, and the dataset will not hand you the answer: the project schema has a retryOverheadPct field, and in the 2026-10-27 snapshot it is null on all 30 projects. What does exist is a workflow attribution for Claude Code, the closest public evidence of where an agent's tokens go.

2. The 31% line in the Claude Code split

Claude Code's row uses the project card figures from the Artificial Analysis Coding Agent Index v2.0 (Sonnet 5.5, max) — benchmarks.json still holds the older v1.5 rows, flagged UNVERIFIED/STALE: 24.1M tokens and $13.30 per task. The same row attributes tokens to six workflow phases. Five of the six are the work itself; one of them is the cost of getting the work wrong.

Debug and retry is the largest phase on both measures: 31% of tokens and 92 of 266 tool calls, or 34.6%. Grouping it with testing makes the picture sharper — test runs and re-runs together are 49% of the bill, more than double the 22% spent editing. The first test run is planned work. The debugging that follows is a tax on the agent's first attempt being wrong.

At this row's prices the retry line is 0.31 × $13.30 = $4.12 per task, or about 7.5M of the 24.1M tokens. That is the amount a cost model built on "tokens per task × model price" quietly folds into a single figure.

One boundary: this is a workflow attribution for one configuration on a benchmark harness, not a metered invoice per phase on production repositories. It establishes the mechanism and magnitude, not your agent's spend.

3. Where the retries come from

The 31% is not one failure mode. It is at least three, and they have different fixes.

Tool failures

The shell returns a non-zero exit code, a patch does not apply because the surrounding lines moved, a test runner times out, an MCP server drops a connection. The agent re-reads the relevant context and tries again. This is the cheapest class per event and usually the most frequent.

Test loops

Write, run, fail, read the traceback, patch, run again. This is where cost multiplies, because the failure evidence — stack trace, diff, surrounding source — is fed back into the context and then re-sent on every subsequent turn. The row shows the shape: 46 tool calls on testing, 92 on debugging, 58 on editing. Each of those calls carries a context window that has grown.

Validation failures and compaction

The harness rejects a malformed diff, a structured output does not match its schema, or the verify step finds the task unsolved. There is also a quieter cost: when the context fills, compaction rewrites history into a summary and charges for the summary, and the next turn has to re-establish state that was previously cached. Compaction is not a retry in the strict sense, but it is spend caused by an attempt running long, and it lands in the same accounting line.

4. How to measure retry overhead on your own runs

Published per-phase splits are rare, so for most teams this has to be measured rather than looked up. The method is unglamorous:

1. Freeze a task set. Ten to twenty tasks with a deterministic pass condition. Without a fixed set, every number is noise.
2. Log attempts, not tasks. Input, output and cached tokens, tool calls and outcome (pass, fail, abandoned) for each attempt. A task that took three attempts is three observations, not one.
3. Tag retry tokens by cause. Tool error, test failure, validation rejection, compaction. A single retry total tells you the size of the problem and nothing about the fix.
4. Pick one convention and state it. Retry overhead is either a share of total spend (31% here) or a multiplier on a clean attempt (0.449 here, since 0.31 / 0.69). The methodology formula uses the multiplier; dashboards usually show the share. Mixing the two overstates the bill by roughly a third.
5. Divide by success rate. Failed runs are paid for, which is what turns a token measurement into a cost per solved task.

Steps 2 and 3 are what the telemetry SDK and OpenTelemetry bridge exist for: post token counts, tool calls, retry causes and outcomes per run, then aggregate against the same frozen task set.

5. What it does to cost per solved task

The methodology prices a solved task in three steps:

blended $/1M = (input_share * input_price) + (output_share * output_price)
             - (cache_hit_share * input_share * (input_price - cached_price))
attempt cost  = (tokens_per_task / 1e6) * blended $/1M
solved cost   = attempt cost * (1 + retry_overhead) / success_rate

Retry overhead enters as a multiplier on the attempt, and the success rate divides. Take the SWE-bench Verified row for Claude Code — 72.8% resolved at $0.85 per task — and vary only the retry multiplier:

The slope is the useful part. Every 10 percentage points of retry overhead adds $0.085 to the attempt and about $0.12 to the solved task at this cost level. Across 30,000 solved tasks — roughly a month at the dataset's modelled run volume, itself a model estimate — that slope is worth about $3,500 per 10 points.

6. Why production is usually worse than the benchmark

Treat a benchmark retry rate as a floor, for structural reasons rather than adversarial ones:

• Task sets are curated. SWE-bench Verified issues are self-contained and reproducible, with a test that fails before the fix and passes after. Production tickets arrive with missing context, drifting dependencies, and specs that change mid-flight.
• Benchmark environments are stable; CI is not. A failing build, an expired credential or a container image that moved turns a code problem into a retry problem, and the agent pays to rediscover that.
• Averages hide re-work. Published cost per task is an average across a task set. It does not tell you what share of the spend went into retries, and the benchmark's own retry-heavy failures are averaged into the same row.

We cannot quantify that gap from public evidence, which is why the dataset leaves retry overhead null for every project rather than filling it with a guess. The practical conclusion is narrow: retry is a line item large enough to change a verdict, it is absent from nearly every vendor quote, and the only honest way to price it is to log your own attempts. Compare published per-task costs on the projects index or the leaderboards, then price your own workload — including a retry multiplier you can defend — in the calculator.

Sources

• Claude Code per-task tokens, cost and workflow attribution — projects.json (claude-code), dataset snapshot 2026-10-27; underlying figures from the Artificial Analysis Coding Agent Index v2.0 comparison.
• SWE-bench Verified resolved rates and per-task costs — TrustTheBench agents leaderboard, cross-checked for the 2026-10-27 snapshot.
• Retry overhead definition, metric definitions and the solved-cost formula — AgentBurn methodology.
• Model list prices and cached-input rates — models.json, snapshot 2026-10-27; list prices only, no negotiated or batch discounts.
• Interactive version of the arithmetic — cost & ROI calculator.