- Most agent projects fail at the approval meeting, not in production — a confident demo hides a business case that was never going to reach the intended outcome.
- Force a named business metric with a known baseline; a project that cannot state today's number cannot prove tomorrow's improvement.
- Treat exception handling — detection, escalation to a named human, and containment of irreversible actions — as the core of the design, not a footnote.
- Decide deliberately what you own versus rent across data, model, infrastructure, and vendor, so a supplier's price or policy change does not become your crisis.
- Approve pilots as bounded experiments with pre-agreed success, stop, and review thresholds, removing the sunk-cost trap before it can form.
Most AI agent projects do not fail in production. They fail at approval — in the meeting where a confident deck, a plausible demo, and an enthusiastic sponsor combine to green-light work that was never going to reach the outcome anyone imagined. The demo works because a demo is a controlled environment. The business does not run in a controlled environment. By the time the gap becomes visible, the budget is committed, the team is invested, and the political cost of stopping is high.
The good news for anyone holding the approval decision is that the failure modes are predictable. Agent projects tend to die for a small number of recurring reasons, and each one can be exposed by a single question asked before money moves. You do not need to understand transformers, orchestration frameworks, or the difference between one model and another. You need to interrogate the business case with the same rigour you would apply to any capital allocation.
What follows is a pre-approval checklist built around five questions. They are deliberately blunt. Each one is framed so that an unconvincing answer is itself the finding: if the team cannot answer clearly, that is your signal, not a detail to resolve later. Treat vagueness as a red flag, not a rounding error.
1. What business metric moves — and by roughly how much?
The first question is the one most often skipped, because it feels obvious and rarely is. Ask which single number on the P&L or in operations is expected to change: cost per case handled, cycle time on a specific process, revenue captured from faster response, hours returned to a constrained team. If the answer is a capability rather than a metric — "it will handle customer queries," "it will summarise contracts" — you have a technology looking for a job, not a business case.
You are not asking for a fabricated forecast. You are asking for a named metric, a rough direction and magnitude, and a way to measure it that already exists or can be stood up cheaply. If the team cannot tell you what to measure, they cannot tell you whether the project worked. And if the honest expected impact is a marginal improvement to a metric nobody in the room cares about, the correct decision is to decline now and spend the attention elsewhere.
2. Is the process stable and rule-describable enough to hand over?
Agents perform best on processes that are understood well enough to explain, yet repetitive enough to be worth automating. The trap sits at both extremes. A process that changes every quarter, depends on undocumented judgement, or lives entirely in one person's head will resist automation regardless of how capable the underlying model is. A process nobody can describe cannot be delegated to a machine, because you have nothing to check the machine against.
A useful diagnostic is to ask the team to write down the process as it actually runs — not the official version, the real one — including the exceptions. Two failure signals appear quickly. First, if the description keeps expanding with "it depends," the process is not yet stable enough to hand over. Second, if the exceptions turn out to be the majority of the volume, you are automating the easy minority and leaving the hard, expensive part untouched.
An agent cannot be more reliable than your understanding of the work you are asking it to do.
3. What happens on an exception — and who catches it?
Every serious agent deployment lives or dies on how it behaves when it is uncertain or wrong. The demo shows the happy path. Production is mostly edge cases, ambiguity, and inputs nobody anticipated. So the question that separates a real plan from a hopeful one is simple: when the agent is not confident, what does it do, and which human picks it up?
A credible answer has three parts, and you should ask for all three explicitly.
- Detection — how the system recognises it is out of its depth, rather than confidently proceeding with a wrong answer.
- Escalation — the named role or queue that receives the handoff, and whether that person has the time and authority to act on it.
- Containment — what the agent is prevented from doing autonomously, so that a mistake is recoverable rather than irreversible.
If the escalation path is "a human will review everything," you have not automated anything — you have added a machine step in front of the same amount of human work. If the path is "the agent handles it end to end" for a process where errors are costly or hard to reverse, you are carrying more risk than the business case admits. The right design usually sits between the two, and knowing where is part of what you are approving.
4. What do we own, and what are we renting?
Agent systems are assembled from parts you own and parts you rent: your data, a model, the infrastructure it runs on, and the vendors who tie it together. Each rented component is a dependency, and dependencies concentrate over time into leverage that someone else holds over you. The question to force into the open before approval is which parts you can change later without rebuilding the whole thing.
The specific exposures worth naming are your data — where it goes, who can see it, and whether using the system trains someone else's product; the model — whether the design assumes one specific provider or can substitute another as prices, terms, and capabilities shift; and the vendor — how hard it would be to leave, and whether your process logic and data would leave with you. You do not need to insist on owning everything; renting is often the right call early. You need to know what you are renting, so that a supplier's price change or policy change does not become your operational crisis. Optionality has a cost, and it is a decision to make deliberately at approval, not to discover under duress.
5. How will we know — within one pilot — whether to scale or stop?
The final question is the discipline that makes the other four safe to get wrong. Approve the project as a bounded experiment with a decision at the end, not as an open commitment that quietly becomes permanent. Before work starts, agree the success criteria, the timebox, and — this is the part teams resist — the conditions under which you will stop.
A well-formed pilot states in advance what result would justify scaling, what result would justify stopping, and what would count as neither so you already know it needs a second look. Tie those thresholds to the metric from the first question. The purpose is not bureaucratic. It is to remove the sunk-cost trap before it forms, so that stopping is a planned, respectable outcome rather than an admission of failure. A pilot you are not genuinely willing to stop is not a pilot. It is an unbudgeted programme wearing a pilot's clothes.
The decision in front of you
These five questions are not a scoring rubric, and a project does not need a perfect answer to every one. What they are is a filter for the specific kind of confidence that gets projects approved and then quietly abandoned. A team that can name the metric, describe the process honestly, plan for exceptions, map its dependencies, and commit to a stop condition has done the thinking that predicts success. A team that answers in capabilities and enthusiasm has not — and that is worth knowing before you sign, not after.
If a proposal in front of you leaves any of the five genuinely unanswered, the most valuable thing you can do is not reject it. It is to send it back with the specific question attached. The next decision is rarely go or no-go. It is whether the case is complete enough to decide at all.
Common questions
Do we need a precise ROI forecast before approving an AI agent project?
No — and a suspiciously precise forecast should make you more cautious, not less. What you need is a named metric, its current baseline, and a rough expected direction and magnitude of change. The discipline is in knowing what to measure and being able to measure it, not in manufacturing a number to the decimal point.
What is the single most common reason these projects fail after approval?
Underestimating exceptions. Demos show the happy path, but production is dominated by ambiguity and edge cases. When there is no clear plan for how the agent detects uncertainty, who it escalates to, and what it is prevented from doing on its own, small errors accumulate into a loss of trust that quietly kills the deployment.
Should we insist on owning the model and infrastructure rather than renting?
Not necessarily. Renting is often the right choice, especially early. The point is to make the decision deliberately: know where your data goes, whether the design assumes a single model provider, and how hard it would be to change vendors later. Optionality has a cost, and you should choose to pay it or not with eyes open — not discover the dependency under pressure.