PerspectiveEnterprise AI · Advanced · 8 min read

Runtime policy enforcement is the job. The policy doc is just paperwork.

Runtime policy enforcement, not documentation, governs AI agents. Writing a rule down is easy. Making an agent physically unable to break it is the work.

Ankur Chrungoo
Share
Series: Enterprise AI Context Layer · Part 5

Most AI governance is a document the agent has never read and cannot be stopped by. Here is the difference between a rule you wrote and a rule that holds.

The last piece ended on a distinction worth a whole article, because it is where most governance programmes quietly fail. Writing a policy down and enforcing it are different engineering problems. One produces a document; the other produces a system that refuses. Runtime policy enforcement (the rule applied in the request path, before the agent acts) is the half that decides whether an agent is safe to run. Confuse it with the document, and you end up with a thick binder of controls and an agent that sails through every one of them.

This is not an abstract worry. It is the single most common way agent programmes deceive themselves.

The most common non-control in the building

Consider how most teams 'govern' an agent today: they put the rules in the prompt. You are a helpful assistant. Do not reveal salary data. Only act on behalf of authorised users. It reads like a control. It is not one. It is a polite request to a system whose entire job is to generate plausible continuations of text, and which will drop the instruction the moment a cleverly worded input, an injected instruction buried in a document it retrieves, or plain probabilistic drift points it elsewhere. A rule that lives only in the prompt is a rule the agent can talk itself out of. Adversaries know this. Auditors are learning it.

Runtime policy enforcement is the opposite posture. The rule does not sit in the agent's context asking to be obeyed; it sits in front of the agent, in a component the agent does not control, deciding what the agent is even able to see and do before it reasons at all.

What runtime policy enforcement actually requires

Real enforcement is unglamorous and specific, which is exactly why it gets skipped in favour of a better demo. A few things have to hold.

The layer has to know who is really asking. That means separating the identity of the principal behind the request (a person, or a service or autonomous agent with a named accountable owner) from the agent's own identity, and binding both to a declared purpose and channel, rather than letting a single over-privileged service account stand in for everyone. Most pilots fail here first: the agent runs as something with far more reach than the principal it acts for, and inherits all of it the moment it is handed a tool.

Decisions have to be made against declared purpose, not role alone, and they have to be least-privilege by construction: deny beats allow, and anything matching no rule defaults to deny. Policy has to be declarative configuration a second-line risk function can read and change, never logic hard-coded in an application only an engineer can find. When a request arrives under an unapproved purpose, the right behaviour is to deny before assembling anything, so there is nothing to leak. The rule fires before the context exists, not after the agent has already seen it. And the layer has to return the least context the task needs, with non-entitled fields stripped from real data in line, rather than relying on someone having pre-redacted a table months ago and hoping it still holds.

MCP is a doorway, not a guard

The protocol conversation makes this sharper. Model Context Protocol is a good standard for connecting agents to tools and data, and it is becoming the default. It is also, on its own, an open doorway. It standardises the plumbing and says almost nothing about authorisation, data classification, input validation, output sanitisation, or audit.

The right move is not to expose your systems over raw MCP and hope. It is to put an MCP-compatible facade in front of them: one that presents approved context and tools only after it has checked identity, purpose, policy, and data classification. Same protocol on the wire, completely different security posture behind it. The facade is where the allow-list of tools lives, where each tool's inputs are validated and its outputs sanitised, where rate and context-size limits apply, where sensitive actions route to a human, and where every request is recorded.

A handful of anti-patterns tell you enforcement has been faked rather than built. Raw database tools handed straight to an agent. Prompt instructions used as the only control. The agent left to decide whether it is authorised, which is like asking the applicant to approve their own loan. Returning more context than the task needs, on the theory that the model will ignore the rest. And failing to record what context the agent actually used, so no one can reconstruct the decision later. Any one of these turns a governance story into a hope.

An agent asked to enforce its own permissions is not governed. It is merely trusted. In a regulated business those are not synonyms.

What the gap looks like in production

Take a composite from a tier-1 lender's collections function. The agent is meant to help handlers propose repayment plans, and the team is careful: there is a detailed policy document, a prompt full of guardrails, and a review meeting where everyone agrees the rules are sound. In testing, a handler asks the agent to 'pull everything on this customer to build the fullest picture', and the agent, doing exactly what it was asked, retrieves account notes flagged for a separate dispute the handler had no basis to see. The policy document said, in plain English, that dispute notes were restricted. The prompt repeated it. Neither was a control. There was no component in the request path binding this handler's entitlements to what got assembled, so the restriction existed everywhere except the one place it mattered, at runtime. The fix was not a better-worded policy. It was moving the rule out of the prose and into the path.

What made the fix cheap, in the end, was that the collections team already had a clean notion of 'purpose' (collections, dispute handling, hardship review) sitting in their case system. Binding the agent's requests to that declared purpose, and denying anything outside it before assembly, took a fortnight of engineering rather than a re-platforming. The lesson the team drew was uncomfortable but useful: they had spent three months refining prompt wording and policy prose that never once stopped a disallowed retrieval, and two weeks on enforcement that did. The prose had felt like progress because it was visible and easy to argue about in a meeting. The enforcement was invisible right up until the moment it said no.

The objection: prompt guardrails are getting good enough

The fair counter is that model-level guardrails and prompt-based controls are improving fast, and a review step catches the rest, so runtime enforcement is a heavy solution to a problem the models will soon manage themselves. There is something to it: guardrails do reduce the rate of obvious violations, and for low-stakes agents they may be proportionate.

But 'reduce the rate' is the tell. A control that works most of the time is, to a regulator, a control that fails some of the time, and the failures are exactly the adversarial and edge cases that matter most. You cannot show an examiner a probability. You can show them a component that made the disallowed action impossible for this principal and purpose, and a record proving it. A human review step helps, but review fatigue is real, and a reviewer approving the tenth plausible recommendation of the hour is not a strong control either. Enforcement is not heavier than the alternative once you count the cost of the alternative failing. It is the thing that lets you stop arguing about probabilities.

What to do differently on Monday

Put three questions to any agent heading for production, and treat a 'no' as a stop.

Does the agent run with its own identity, bound to a specific accountable principal (a person, or a service with a named owner) and a declared purpose, or does it borrow an over-privileged service account? Is policy enforced before context is assembled, so there is nothing to leak, rather than filtered out of the response afterwards? And does the layer, not the agent, write an exportable record of what it retrieved and why? If any answer is no, you do not have runtime policy enforcement; you have a policy document and an optimistic prompt. That can still reach production, and plenty of agents have, but it cannot prove what the agent did, replay a decision, or hold up where the work is regulated and mistakes are costly. Where that is the bar, closing the gap is the job that remains.

Where this leaves us

None of this is exotic. It is the established engineering that makes any system secure and auditable (identity, least privilege, mediation, validation, audit), applied to a new kind of actor that happens to be fluent, fast, and easily talked into things. Plenty of organisations already run agents in production without it. The difference is not whether the agent ships; it is whether, in a regulated or high-risk setting, you can prove what it did, replay how it decided, and defend it when someone asks. Move the rules out of the prose and into the path and you can. Leave them in an excellent document and you cannot.

Enforcement has to live somewhere concrete, though. It is time to look at the structure that carries it.

Next: how to build a context layer that holds up under audit, and the one ordering decision the whole design turns on.

Was this useful?
Share

The Engineering Notebook

Once a month, a long read on what we're learning building governed AI for regulated enterprises. No hot takes, no roundups.

Prefer to talk it through?

Ankur Chrungoo

Principal Engineer and Architect

Principal Engineer and architect at Bugni Labs, writing about production AI systems, agent governance, model controls, and regulated decisioning.

Related case studies