PerspectiveEnterprise AI · Advanced · 10 min read

Enterprise AI agents don't have a model problem. They have a context problem.

Enterprise AI agents keep stalling before production. The blocker is not the model. It is trustworthy context and control enforced the moment an agent acts.

Ankur Chrungoo
Share
Series: Enterprise AI Context Layer · Part 1

Why agentic AI keeps stalling on the way to production, and the unglamorous half of the answer that most teams skip.

Here is a pattern you can set your watch by in almost any large regulated business now building enterprise AI agents. A team stands one up in a fortnight. It reads a few systems, drafts a decision, and looks convincing in the boardroom. Someone says 'transformational'. A budget appears.

Then it sits there. Not killed, not shipped. Six months on, the model underneath has been upgraded twice, the demo looks identical to the one from the spring, and the thing still has not gone live to a single real customer.

The usual explanation is that the model is not ready, or the framework is immature, or that 'governance' is dragging its feet. I think that is mostly wrong, and the industry has spent the better part of a year paying for the mistake. The models crossed the threshold a while ago. What enterprise AI agents lack is context they can trust, and control that actually holds at the instant they act.

That is the argument of this piece, and of the series it opens. The bottleneck between a pilot that impresses and a system you can run is not intelligence. It is a governed context layer: the tier that gives an agent the right meaning and evidence to reason over, and that enforces who may see and do what before the agent acts. Get that layer right and the choice of model almost stops mattering. Get it wrong and no model, however good, gets you past your own risk committee.

The day the stakes changed

For two years, enterprise AI mostly wrote things. A bad sentence was a bad sentence; you edited it and moved on. The failure mode was embarrassment, and embarrassment is cheap.

An agent does not write, it acts. It reads records, calls tools, moves money, updates a case, closes a ticket. The failure mode is no longer an awkward paragraph. It is the wrong customer's account touched. It is a field the person behind the request was never cleared to see, surfaced anyway. It is an irreversible step taken with nothing on file about who authorised it or why.

The moment AI stopped talking and started doing, the hard part stopped being language. It became authority, and proof. Those are the two words a regulator asks about, and neither of them lives in the model.

What enterprise AI agents actually lack

Point an agent at your warehouse and watch it rediscover the schema on every prompt. It will guess the join key, pick a plausible date filter, and decide for itself what 'active customer' means, differently each time, always with total confidence. Ask two agents the same question and you can get two answers, both citing your own numbers. Neither is lying. Nobody told them which definition was the sanctioned one, because that definition mostly lives in a dashboard someone built in 2021 and in the heads of three analysts who have since moved teams.

Then there is access. Most pilots I have seen run on a single service account with far more reach than any individual user should have; the agent inherits all of it the moment you hand it a tool. It works beautifully in the demo. It becomes an audit finding the first time someone asks who could read the salary table, or the vulnerable-customer notes, or the full history of an account the investigator was never assigned.

Take a composite drawn from work in regulated lending. A tier-1 UK lender's financial-crime team builds an agent to help investigators triage suspicious activity. In the demo it is superb: it summarises a case, pulls the transaction history, proposes a next action, and does it in seconds against a queue that used to take an analyst half a morning. To ship it, the second line of defence asks four questions. Which investigators can invoke it, and for which cases? When it reads a customer record, is it applying the same entitlement rules a human would? When it recommends freezing an account, who signs? And when the model is wrong, what is the record that lets us reconstruct why it did what it did?

The pilot answered none of the four, because all four live below the model, in a layer nobody had built. The agent was not short of intelligence. It was short of context and control. That gap, between the demo everyone applauds and the system the second line will sign, is not a modelling problem you can wait out. It is plumbing, and plumbing does not build itself while you upgrade the engine.

The market is naming the same gap

The useful signal is where this is showing up. The major data platforms are now shipping governed semantic objects built specifically so an agent has something trustworthy to stand on, and a distinct context layer (the tier sitting between enterprise data and the reasoning above it) is becoming a named part of the stack rather than an abstraction. The people closest to the agent protocols keep pointing at the same hole: MCP and A2A are good at moving context around, and say almost nothing about who is allowed to see it, or how you would later prove what happened.

Connectivity is turning into a commodity; there is a connector for everything now. The part still largely unsolved, recognised widely as the need, but not yet delivered in a neutral, enterprise-owned form, is the governance that has to sit on top of the connectivity. That is the ground this series stakes out.

Connectivity is becoming a commodity. The governance above it is the part still unsolved, and the part actually worth owning.

Two jobs, on every request

What closes the gap is a thin layer that your enterprise owns, sitting between the agents and the systems, doing two jobs each time an agent asks for something.

It assembles the meaning, the know-how, and the evidence the task genuinely needs, so the agent reasons over your business the way your business actually defines itself rather than the way it guessed. And it enforces, in line and before the agent reasons, who may see and do what for this user, this purpose, this request; then it returns a record you could put in front of a regulator without flinching.

It does not replace what you run today. Your data stays where it is. The layer federates over the estate you already have and hands each agent only what it is entitled to, in a shape it can act on safely. Framed in the way we build at Bugni, this is an AI-Native Engineering concern rather than a policy one: the control belongs in how the system is validated and operated, not in a governance document the agent will never read.

Knowing a rule is not the same as enforcing it

This is the distinction most tools blur, and getting it right decides everything else.

Recording that analysts cannot see certain fields is easy; any catalogue holds rules like that. Making sure an agent physically cannot pull those fields, for this person and this purpose, and being able to show afterwards that it did not, is the actual work. It is also the expensive, unglamorous work, which is why so many products stop at the first half and call it governance.

A context foundation documents. A context-and-control plane enforces, and produces the evidence that it enforced. In a regulated business, that line is the whole difference between a promising pilot and something operations will sign.

The obvious objection: won't the platforms just absorb this?

The strongest counter to all of this is that the problem is temporary, and the platforms will swallow it. The data clouds are already adding governed semantics and agent controls; the model vendors are adding memory and tool permissions. Wait eighteen months, the argument runs, and the context layer becomes a feature of something you already buy rather than a thing you build.

There is real truth in it, and it would be dishonest to wave it away. Some of this will be absorbed, and the semantic half (the governed meaning an agent reasons over) is being commoditised in front of us. If your only need is consistent definitions inside one data platform, you may well get that from the platform.

But two things do not absorb cleanly. The first is enforcement that spans systems you did not buy from a single vendor. A bank runs a warehouse from one supplier, case management from another, a fraud platform from a third, and a fistful of bespoke services underneath. Entitlements, purpose-binding, and one coherent audit trail have to hold across all of them, at the moment of action, or they do not hold at all. No single platform owns that seam, because owning it would mean owning your entire estate. The second is portability. The organisations that think hardest about this do not want their context, their policies, and their audit trail sealed inside the one vendor they can never leave. The control plane is precisely the part they will want to keep in their own hands. So buy the semantic layer where it is genuinely good; the enforcement and evidence above it is the part worth building and owning.

What to do differently on Monday

If you lead this, the useful first move is not to pick a model or a framework. It is to make the layer explicit and hold your teams to it.

Start by pulling the two halves apart on paper: what the agent is allowed to know, and what it is allowed to do and prove. Most stalled pilots have quietly conflated the two, which is exactly why they cannot answer the second line's questions. Then put three tests to any agent heading for production. Does it run with its own identity, bound to a real user and a declared purpose, or does it borrow an over-privileged service account? Is policy enforced before context is assembled (so there is nothing to leak) rather than filtered out of the answer afterwards? And does the layer, not the agent, write an exportable record of every run? If any answer is no, the pilot is not close to production, whatever the demo looked like.

The uncomfortable part for a leadership team is that none of this is a model upgrade you can procure. It is engineering: identity, entitlements, mediation, and evidence, applied to a new kind of actor that happens to be fluent, fast, and easily talked into things. That work is unglamorous, and it is precisely the work that separates the enterprises that will run agents in front of customers and regulators from the ones still admiring their pilots a year from now.

Where this actually gets won

The enterprises that come out ahead in the agentic era will not be the ones with the best model. Everyone will have the best model; it will be a line item. They will be the ones who can put an agent in front of a customer, or a regulator, and show their working: that it was authorised, that it was grounded in real evidence, that a human stood behind the decisions that warranted one.

That is a context problem, not a model problem. It is worth solving properly, and unlike two years ago, it is solvable now.

Next: what a context layer actually is, and why the half everyone skips is the half that decides whether an agent ships.

Was this useful?
Share

The Engineering Notebook

Once a month, a long read on what we're learning building governed AI for regulated enterprises. No hot takes, no roundups.

Prefer to talk it through?

Ankur Chrungoo

Principal Engineer and Architect

Principal Engineer and architect at Bugni Labs, writing about production AI systems, agent governance, model controls, and regulated decisioning.

Related case studies