the production gap

Two numbers from the same year that don’t seem to fit together. In 2025, according to the OECD, 61 percent of all venture capital in the world went to AI, more than 258 billion dollars. That same year, Gartner predicted that more than 40 percent of agentic AI projects will be scrapped before the end of 2027, and MIT concluded that 95 percent of generative AI pilots deliver no measurable impact on the bottom line.

So much money, so little of it reaching production. It looks like a paradox. It is the opposite: it is exactly the story I watch up close every day. There is a gap between an agent that impresses and an agent that works, and almost no one spending the money has crossed that gap themselves.

a demo and a service are two different crafts

A demo has to work once, while everyone is watching. A production system has to behave while no one is watching. That is not wordplay, it is a different craft.

In the demo the builder picks the input, runs the path he knows, and stops the moment the applause starts. In production the world picks the input, the path is different every time, and nothing stops on its own. A chief operating officer in manufacturing put it drily in the MIT study: “The hype on LinkedIn says everything has changed, but in our operations, nothing fundamental has shifted.” On stage, everything has changed. On the floor, nothing yet.

the arithmetic nobody puts on the slide

There is a simple reason why demos succeed and production fails, and it is arithmetic.

Suppose an agent does the right thing 95 percent of the time at each step. That is optimistic for today’s models. A three-step demo then almost always gets away with it. But reliability across a chain is a multiplication, not a sum. At twenty steps the odds of a flawless run drop to roughly one in three. A production process worth building is rarely three steps. It is twenty or thirty.

This is a simplification, real errors are not neatly independent. But the direction is right, and every builder feels it. More autonomy means more steps, and more steps means one weak link drags the whole chain down with it. I saw it myself when my own engine ended up in a loop one evening in May that fed on itself. The mistake looked perfectly reasonable in review. That is the treacherous part: in the demo, it was invisible.

it is not a model problem

The reflex is to blame the model, and to look for the fix in a smarter model. Both miss.

The expensive mistakes of the past year were not a model being stupid, they were actions without a brake. When Air Canada’s chatbot invented a bereavement discount that did not exist, the tribunal ruled that the company was on the hook for it: it makes no difference whether the information comes from a static page or a chatbot. When Replit’s coding agent wiped an entire production database during an explicit freeze, it confessed afterward to “a catastrophic error in judgment,” and then produced thousands of fake records that masked its own mistake. A chatbot that hallucinates lies to a user. An agent that hallucinates does something. It refunds the wrong customer. It deletes a row that was supposed to stay.

The causes run deeper than the model, and they are tougher. Context rot, for one: the more you feed a model, the worse it performs, often long before the window is full. Anthropic says it about its own models without hedging: treat context as a limited resource with decreasing returns. Every extra token is not free, it eats into the attention the model has left.

This is the lesson my guardrails piece already carries, now confirmed by an entire industry full of wreckage: safety does not live in the cleverness of the model, it lives in the discipline you build into the tooling. A builder running agents across dozens of document types said it better than I can. In a regulated domain, the most constrained agent is the most reliable one.

where it does pay off

This is not a piece against agents. I build them, they run, they deliver. But they deliver in a place less exciting than the market promises.

Agents do not win by reasoning freely. They win by executing a well-documented process a human used to do by hand. The builders who actually pull it off all say a version of the same thing: the system that works is not the smart system, it is the bounded one. Klarna is the sharpest example of this, in both directions. The company had an AI assistant do the work of 853 employees and saved tens of millions with it. And it publicly walked part of it back, with the chief executive admitting: we focused too much on efficiency and cost, and the quality suffered for it. Both true. The win is real, and so is the boomerang.

There is a more optimistic signal too, and it is not nothing. Google’s own study last autumn found that more than half of the executives surveyed say they have agents in production, and that three quarters report a return within the first year. The most mature category is code. But even there the honest number is not triple productivity. The rigorous DORA research found that an individual developer gets tens of percent faster, while the delivery speed of the whole organization can actually fall if the foundation is weak. AI makes a good process better, and a bad process broken faster.

the market disagrees with itself

Notice how often a big number gets quoted without anyone asking what it measures. Estimates of the market size vary by a factor of three to four, purely depending on how narrowly or broadly you define “agent.” Gartner measures something different from MIT, which measures something different again from the optimistic venture capitalists. They are not contradictory, they are counting different things. Anyone who lines up the headlines without the definitions is mistaking noise for insight.

And part of what gets sold as an agent is not an agent at all. Of the thousands of vendors calling themselves agentic, Gartner estimates around 130 are real. The rest is an existing chatbot with a new name. Agent washing, they call it now.

you close the gap with boring discipline

The gap does not get closed by waiting for a smarter model. Andrej Karpathy, who knows what he is talking about, calls robust agents not a one-year project but possibly a decade-long problem. He would rather describe the models as ghosts than as animals: summoned, statistical, with human quirks and very inhuman blind spots. You do not build reliability by politely asking such a ghost to be careful.

You build it with boundaries that sit outside the agent’s reasoning loop. A hard brake that cannot be talked around, because the tool allows nothing else. An explicit threshold for production. A log of every action. A kill switch after a run of failures. None of it is exciting, and that is exactly the point.

The test I hold against every agent plan, mine or a client’s, is the same one from the end of my engine story. Not: what can the system do? But: what happens when the system is wrong, and which line of code stops it then? Whoever can answer that stands on the right side of the production gap. Whoever cannot has no system. They have a demo that has not been allowed to fail yet, and a place in the forty percent.