Automated Decisioning Is a Governance Choice

The instinct, once an institution has a capable model, is to let it decide. Hand it the case, let it weigh the factors, and take back a verdict: clear or block, approve or decline. It looks like the obvious payoff of the investment. For a class of decisions that a bank makes every day, it is the wrong instinct, and this is the sharpest lesson I take from building automated decisioning: the reason has nothing to do with whether the model is good. It has to do with what an institution is actually asked to produce when it makes a consequential decision about a customer.

A verdict is not the deliverable. The deliverable is a decision you can explain, reproduce, and defend, months later, under scrutiny, when a regulator or an ombudsman or a court asks what you knew at the time, which policy you applied, why that produced this outcome, what the human reviewer did, and whether you can reconstruct the whole thing exactly. A decision that cannot answer those questions has not saved you work; it has created a liability that looks like efficiency.

So the position is this. For high-stakes, regulated, reviewable decisions, the durable asset is not a smarter decider but the explainable, versioned record around the decision, and the surprising consequence is that at the precise moment of decision you often want the model nowhere near it.

Why determinism belongs in automated decisioning

I designed and built a decision-intelligence platform for screening workflows, the clear, review, or block family that includes sanctions and watchlist matching and adverse-media review. It was real, working code, engineered around one deliberate choice: what actually made the decision. It was not the model. It was a deterministic evaluator: take the structured signals, apply a versioned policy, and resolve to an outcome. A name-match score against a trusted list, a partial date-of-birth conflict, a source-reliability grade become explicit factors, and a review threshold and a distinct, higher block threshold turn those factors into clear, review, or block.

Keeping that step deterministic was not a vote of no confidence in models generally. It followed from a property the decision has to have and a sampled generative model, when sampling is enabled or provider behaviour is not pinned, does not. Ask a probabilistic model the same question twice and you may get two answers. That variability is a strength when you want range and nuance. It is a liability when the decision must come out the same way every time and be reproducible on demand years later. The reason to keep the model out of the verdict is not that the model is untrustworthy in general. It is that this specific step must be replayable, and a component whose runtime behaviour can vary is the wrong tool for a step whose defining requirement is that it does not.

What determinism actually buys

In that platform the payoff was concrete, because the reproducibility was built rather than promised. Each decision preserved the policy metadata it had used, the thresholds, the rule identifiers that fired, a checksum, and the effective policy version, so the outcome was a function of stored inputs and a named policy snapshot. On top of that sat two capabilities I implemented as first-class operations. One replayed a case, re-running the evaluation to the identical result against the exact policy version in force at the time. The other took the same stored inputs and applied a different policy version, showing what the outcome would have been without disturbing the original case, which is how an institution answers whether tightening a rule last quarter would have caught more cases. Each derived factor also carried its lineage, the inputs and the policy references it came from, so the explanation was the mechanism that produced the outcome rather than a narrative composed after the fact.

Set against the direction of regulation, none of this is decorative. The PRA's model risk management principles push banks towards defensible inventories, governance and evidence of what drove an outcome; the EU AI Act establishes documentation, logging and human-oversight duties for high-risk systems. Consumer-protection duties also ask institutions to show that outcomes were fair and explicable. A deterministic core with a versioned, replayable record is not a nostalgic retreat from AI. It is the part of the design that lets you satisfy all of that, which is why a re-runnable evaluator was worth keeping even where executing the decision was otherwise optional: without it, counterfactual and audit work are not possible at all.

The result resembles the governed record in Bugni's credit decisioning work: inputs, rules, reasons and human action remain available after the model or policy changes.

The steelman, taken seriously

The strongest counter-argument deserves more than a nod, because it is genuinely strong. Deterministic thresholds are not automatically fairer, more accurate, or even more transparent. A threshold is a cliff: a case just the wrong side of the line is blocked while a near-identical case just the right side clears, and that discontinuity can be arbitrary at the margin in a way a model's graded judgement would not be. Rules can be gamed once they are known. They ossify. And a dense thicket of interacting thresholds can be every bit as inscrutable as a neural network while merely looking legible because it is made of numbers a human could in principle read.

All of that is true, and it forces the claim to be precise rather than triumphant. Determinism buys reproducibility, not correctness. It guarantees you can reproduce and explain the decision; it does not guarantee the decision was good. Conflating auditable with right is the same category error as believing a model's fluent rationale means it reasoned well. So the defensible position is narrow and conditional. Keep the model out of the verdict when the cost of an unexplainable error is high and the decision is consequential or hard to reverse, and prefer the property you can prove, reproducibility, over the one you can only hope for, a marginally better guess. Flip those conditions, to low stakes, easily reversed, enormous volume, and letting the model decide while absorbing some error is frequently the better and cheaper choice. This is an allocation of authority to the layer that fits the decision, not a moral stand against models.

The influence that sneaks back in

There is a second-order effect that quietly undermines the tidy version of keeping the model out of the decision, and it is the one senior leaders most often miss. You do not actually keep the model out simply by keeping it out of the resolution step. The natural place to put a model is summarising the case for the human reviewer, and that human is usually the final authority. If the model's summary chooses which facts lead, what gets emphasised, and what tone to strike, it is shaping the decision through the reviewer's anchoring, quietly, by way of automation bias. Moving the model from decider to summariser-for-the-decider reduces its authority; it does not remove it. It launders the influence through a person who now feels as though they decided.

The same applies when a deterministic evaluator consumes a model-derived factor. Determinism at the final step does not erase upstream model influence. Those inputs need to be stored, validated and exposed in the explanation so the replay includes the conditions that shaped the verdict rather than only the last rule that fired.

For an institution relying on a human-in-the-loop as its control, this is the uncomfortable question that must be asked honestly: is the reviewer genuinely weighing the case, or endorsing what the model implied? If you are serious about the model not deciding, you have to design the reviewer's view to resist the pull, showing the structured factors first, keeping the model's prose clearly secondary, and actually measuring whether reviewers diverge from the model's steer or simply follow it. Otherwise you have built the appearance of human judgement over the substance of model judgement, which is worse than either honestly chosen, because it fails an audit while looking like it should pass.

What to do differently on Monday

Draw the boundary of your decisioning products around the governed record, not the cleverness of the engine, because engines get swapped and records get audited, so the thing to invest in is the thing that gets audited: the inputs, the policy version, the factors, the proposed outcome, the human action, and the ability to replay. Reserve the model for the work where variability is acceptable, retrieval, drafting, summarising, surfacing patterns, and be conservative about placing a sampled model at any point that must reproduce identically on demand. Version your policies and store your inputs from the outset, because replay and counterfactual analysis fall out almost for free when you do and are close to unrecoverable when you do not. And watch the reviewer's screen as closely as the decision engine, because that is where the model's influence actually re-enters, and where a human-in-the-loop control quietly turns into theatre if nobody is measuring it.

Make these authority boundaries explicit in the AI-Native Engineering governance model, where model behaviour, deterministic policy and human accountability can be tested separately.

Govern the record, not the demonstration

The demonstration that impresses a boardroom is the AI that decides. The system that survives an audit is the one that can prove, step by step and the same way every time, how a decision was reached, and that was deliberate about which layer held the authority. Ask not only whether your model is accurate, but whether, three years from now, you could reconstruct exactly why a customer was declined, and whether the person who signed it off was truly deciding or merely agreeing.