How to Measure the Economics of an AI Agent

Most agent business cases fall apart in the same place. Someone quotes a strikingly low number — a fraction of a cent per thousand tokens — and concludes that automating the work is almost free. Then the pilot ships, and the real bill arrives: repeated model calls per task, retrieval and orchestration, a queue of human reviewers checking the agent's output, and the slow, expensive cleanup when it resolves something confidently and wrongly. The token price was real. It was also almost irrelevant.

The problem is the denominator. Cost per token measures the cheapest and most volatile input in the system, not the thing the business actually buys. A business does not buy tokens. It buys resolved tasks: a refund processed correctly, an invoice matched, a support case closed, a contract clause flagged. If you cannot state what one resolved task costs you today, and what it would cost with an agent in the loop, you do not have an economic case. You have a demo.

This is a finance problem before it is a technology problem, which is good news for the people usually left out of these decisions. A CFO who can build a unit-economics model for a delivery route or a call centre can build one for an agent. The inputs are unfamiliar; the discipline is not. What follows is a framework for doing it honestly, without a single number we cannot defend.

Start with the right denominator: fully-loaded cost per resolved task

Fix the unit first. The unit is one resolved task, at an agreed quality bar, measured against the process you run today. Everything else is an input to that number. Two words in that sentence do the heavy lifting. Resolved means the outcome was correct and complete, not merely produced — an answer the agent got wrong is not a resolved task, however cheaply it was generated. Fully-loaded means every cost required to reach that outcome, not just the visible model call.

A defensible fully-loaded cost has five components. Estimate each one; do not let the invisible ones default to zero.

1. Model inference — every call needed to resolve one task, including retries, reasoning steps, and tool calls, not a single idealised prompt.
2. Infrastructure and orchestration — retrieval, vector stores, queues, logging, monitoring, and the compute that surrounds the model.
3. Human-in-the-loop overhead — the review, correction, and escalation time a human spends per task, priced at their real loaded rate. This is usually the largest hidden cost.
4. Build and maintenance, amortised — the engineering to create, evaluate, and keep the agent working as models, data, and processes change, spread across the volume it handles.
5. Rework and error cost — the downstream cost of the tasks it gets wrong, which deserves its own section below.

Price the errors, because they set the ceiling

The line item that quietly decides most agent economics is rework. A human agent who is unsure tends to signal it — they ask, they hedge, they escalate. An agent can produce a fluent, confident, wrong answer at the same cost as a right one. That asymmetry is the whole game. The cost of an error is rarely the compute that produced it. It is the refund issued incorrectly, the customer who churns, the compliance exposure, the human hours spent finding and unwinding the mistake.

So the model needs two rates, not one: how often the agent is wrong, and what a wrong answer costs when it happens. A high error rate on low-stakes tasks may be perfectly economic. A low error rate on high-stakes tasks may not be, if each failure is expensive enough. This is why the quality bar has to be set before the pricing, and why the review overhead and the error cost move together — spend more on human review and you cut error cost but raise per-task cost. The right balance is an economic choice, and it belongs to the business, not the model.

An agent's cheapest output and its most expensive mistake can cost exactly the same to generate. Only the business knows which one it just paid for.

Count the value that per-unit cost misses: throughput and availability

Cost per resolved task tells you whether an agent is cheaper. It does not capture two other sources of value, and leaving them out understates the case. The first is throughput and capacity. Human capacity scales roughly linearly — more volume means more headcount, hiring lead time, training, and management. Agent capacity scales differently: it absorbs demand spikes without a proportional cost step, and it does not need to be rebuilt each time volume doubles. For work with sharp peaks — seasonal demand, launch surges, month-end — the value is often in the peak you no longer have to staff for, not the average unit cost.

The second is speed and availability. Continuous operation and near-instant response have real economic value where latency changes outcomes: faster resolution that prevents a customer defecting, an approval that clears before a deadline, a discrepancy caught before it propagates. This value is genuine but easy to overclaim. State it qualitatively, tie it to a specific outcome you can point to, and resist the urge to invent a precise figure. A reasoned, conservative statement is more credible to a board than a fabricated percentage — and it survives scrutiny.

Build the model yourself, on a narrow case

You do not need a data-science team to build this. You need one well-chosen task and honest instrumentation. Pick a single, high-volume, well-understood task where you already know the baseline cost, time, and error rate. A narrow use case with a clean resolution definition will teach you more than a broad one with a fuzzy one, because the entire model rests on being able to say, unambiguously, when a task is resolved.

1. Define resolution — write down exactly what a correct, complete outcome is, and how you will verify it. If you cannot verify it, you cannot cost it.
2. Measure the baseline — the fully-loaded cost, time, and error rate of doing this task today, with people or existing software.
3. Instrument the agent — capture calls, human review minutes, escalations, and errors per task, on the same definition and the same quality bar.
4. Compute fully-loaded cost per resolved task — total cost over tasks genuinely resolved, then set it beside the baseline.
5. Layer in throughput and availability value — stated conservatively and tied to specific outcomes, not invented figures.

Run this and one of three things becomes clear. The agent is decisively cheaper on a fair, fully-loaded basis, and you scale with evidence. It is roughly break-even, and the case rests on throughput, availability, or freeing scarce people for higher-value work — a real argument, but one to make explicitly rather than smuggle in. Or it is more expensive once rework is counted honestly, which is worth knowing before you have committed budget, not after.

The next decision

The economics of an agent are knowable before you commit at scale, and the cost of finding out is small: one task, one baseline, one honest model. The failure mode is not building the model badly. It is skipping it — being seduced by a token price into a build with no defensible denominator, or dismissing agents on the same flawed number pointed the other way. Before the next agent proposal reaches your desk, ask for one thing: the fully-loaded cost per resolved task, and the baseline it is measured against. If those two numbers are not in the room, the conversation is not ready to have. If they are, you can decide the way you decide everything else — on the unit economics, with control over your data, models, and vendors intact, and with your eyes open.