Ask a room of operators how many AI pilots their company is running, and you get a big number. Ask how many are in production, owned and run by a team, with someone accountable when they fail, and the number drops by an order of magnitude. Ask how many AI projects and agents were retired on purpose, rather than quietly abandoned, and you usually get a laugh.

This is not just a maturity problem. Enterprises have been doing this for four years now. The ones stuck in pilot purgatory are not less sophisticated than the ones that ship and generate ROI. They run a process that produces pilots, and it works exactly as designed.

A pilot is not a small production system

The category error is at the beginning, and it is so common it barely registers as a choice.

A pilot is built to answer one question: can the technology do this at all? So you scope it to the cases where the answer is most likely to be yes. Clean data. A cooperative team. The happy path. You exclude the messy 20% because the messy 20% is not what you are testing. Six weeks later you have an answer, and the answer is yes, and everyone is pleased.

Production answers a different question: can this run every day, on everything, including the cases you excluded, when the person who built it has moved on? Those two questions have almost nothing to do with each other. The first is about capability. The second is about reliability, ownership, and failure.

So the pilot succeeds, and the thing it proved turns out not to be the thing you needed proved. The team asks for production funding. The funder, reasonably, asks what changed since the successful pilot. The answer is the scope, the population, the failure modes, the support model, and the cost, which makes it sound like the team is moving the goalposts. Nobody is lying. The pilot was never a small version of the system.

What was the demo hiding?

Three costs sit entirely outside a typical pilot budget, and all three land the moment you try to graduate.

The exception population. The 20% you excluded is not 20% of the effort. In every operational process I have looked at closely, the exceptions carry most of the handling time and nearly all of the risk. A pilot on the happy path tells you the easy work can be automated, which you suspected. It tells you nothing about whether the hard work can be, and the hard work is where the money is.

The ownership line. A pilot is owned by whoever built it, usually an innovation team or a vendor. Production needs a name, an on-call rotation, and a budget line that survives the next reorg. In most organizations, no such team exists for agents, so graduation requires creating one, and creating a team is a much harder ask than approving a project. This is the single most common place I see a working pilot die: not rejected, just unassigned.

The off-ramp. Nobody asks a pilot what happens when it fails, because when it does, you switch it off and everyone goes back to what they were doing. In production, the manual capacity you displaced is gone, the people who knew the process have been redeployed, and switching off means the work stops. A production system needs a tested revert path and enough retained human capability to absorb the load. That capability costs money to keep and produces nothing on a good day, so it’s the first thing cut, and that’s why the second failure is worse than the first.

Pick the use case backward

The standard prioritization exercise scores candidate use cases on value and feasibility, picks the top-right quadrant, and starts. It reliably selects for demo quality, because feasibility-in-a-pilot is what the scoring can actually see.

Score the other end instead. For each candidate, ask what it would take to run this on an ordinary Tuesday in eighteen months:

  • Who is on call when it misbehaves at 4 p.m. on a Friday?
  • What is the exception rate, and who handles the exceptions?
  • What does the revert look like, and when was it last tested?
  • What decision does this make that someone external could challenge, and can we show our work?
  • If the vendor disappeared, how long would it take us to notice?

Watch what those questions do to a shortlist. The invoice-coding candidate, which looked merely solid on the old matrix, usually survives: the exceptions are bounded, finance already has an on-call rotation, and reverting means a queue gets longer. The customer-facing triage agent, which topped the old matrix because it demos beautifully, usually does not: nobody wants the Friday shift, the exception rate is unknown because the hard cases never reached the pilot. The revert is “hire back the team you just moved.” Same two candidates, opposite ranking, and the second ranking predicts what you will still be running in two years.

A use case that survives those five questions is worth piloting. One that does not is worth a conversation, not a pilot. This inverts the usual sequence, and it is slower at the start, which is precisely why it is unpopular and precisely why it works: the constraint on your AI program has never been the number of things you could try.

Write the graduation gate down first

The practical change is small and almost administrative. Before a pilot starts, write down what it must demonstrate to become a system, and who signs.

Not did it work. Specifically: the outcome quality it must hold on the full population including exceptions, the cost per task it must beat, the named owner it graduates to, the revert procedure it must have tested at least once, and the review it must pass to make decisions a regulator or a customer could contest. Five lines. Agreed in advance, by the person who will fund production, not the person running the pilot.

Two things happen when you do this. Some pilots never start, because writing the gate reveals that nobody is willing to own the result, and that is a finding worth six weeks of anyone’s time. The ones that do start are built against the real bar from day one, which mostly means including the ugly cases early, when they are still cheap to discover.

You also get permission to kill things. A pilot with no written gate can never formally fail, so it does not get retired; it gets deprioritized, keeps its integrations, keeps its cloud bill, and joins the sediment. Half the value of the gate is the ability to say this one did not clear it and mean something by it.

What is the pilot still for?

So many pilots stall not because enterprises are bad at AI, but because the pilot is a genuinely good instrument for answering a question most organizations stopped needing to ask around 2024.

We know the technology works. What we do not know, for any specific process in any specific company, is whether the organization can run it: whether someone will own it, whether the exceptions are tractable, whether the failure path is survivable, whether the economics hold once you count the humans still required. Those are the open questions now, and a pilot is not designed to answer any of them.

I think the companies pulling ahead are not the ones with better models or more pilots. They are the ones that started treating the graduation criteria as the beginning of the work rather than the end.