What 6 billion spans taught us about agent latency

We looked at a full month of production traces — a little over six billion spans — and asked one question: where does the time go in a real agent run?

Retrieval is the tax nobody budgets for

Across the fleet, retrieval and reranking account for a median of thirty-one percent of end-to-end latency. Generation is thirty-eight. The remaining third is tool calls, serialization, and waiting on your own services.

The teams with the fastest agents were not using faster models. They were making fewer round trips — batching retrieval, caching reranks, and refusing to let the agent call a tool it could have answered from context.

Retries are where p99 goes to die

A single retry roughly doubles the run. Two retries on a chain of three tools produces the tail latencies that make people say agents are slow. Cap retries per run rather than per call, and make the cap visible in your traces.

The uncomfortable finding: half the retries we sampled were retrying a call that had already succeeded, because the agent could not tell.