Ask a risk owner at a large regulated company what is stopping them from putting AI agents on real decisions that impact the business, and you will probably expect to hear about regulation. That is the answer the market assumes, but not the answer I'm getting.
What I hear instead is that they want to use AI to make or save millions, but they cannot govern autonomous agents at scale.
The gains are not speculative
The use cases are already named and already budgeted:
- Claims decisions in minutes instead of days.
- Underwriting that returns an answer while the customer is still on the page.
- Support that resolves rather than routes.
In insurance these are not efficiency projects, they are survival projects to stay competitive. The people I talk to are explicit that smaller, faster competitors will take the business if the incumbents wait too long to adopt autonomous agents.
The pull is real. The board has usually already asked for the number.
The blocker is not model quality
Nobody has told me the models are not good enough. The failure is downstream of the model entirely.
When an agent takes a consequential action, someone eventually has to answer a question about it. What did it do? Under what policy? Who allowed that policy? What was the state of the system when it decided? Show me.
Answering that once, for one action, in a postmortem is easy. Answering it for hundreds of thousands of actions a day, on demand, to someone who does not work for you, is the part nobody has solved. That is what "we cannot govern it at scale" actually means. It is not a statement about controls existing. It is a statement about proof existing.
Proof ranks above regulatory uncertainty. Regulatory uncertainty is a spending-timing problem: you do not commit to a two-year, tens-of-millions rollout while the rules are still moving, because you may have to unwind it. The governance gap is different. It does not resolve on its own when the rules settle.
So autonomy gets rationed
The result is a pattern that is easy to spot once you know to look for it.
Autonomy is permitted exactly where being wrong is inconsequential:
- Constrained FAQ bots.
- Retrieval and routing.
- Drafting that a person then signs.
Meanwhile every decision with money or a legal consequence attached keeps a human in the loop. This is deliberate, as policy, not as a technical limitation.
That split is not caution about AI, but a caution about evidence. A company will let an agent do anything that can be easily explained later.
The missing piece is a record, not a control
Most organizations respond by adding more controls: more policy documents, more review boards, more approval steps. Those are worth having but they do not close the gap. A control you cannot prove is indistinguishable from a control you do not have.
What is missing is the record with three properties that ordinary logs do not have:
- Durable. The action, the policy it was evaluated against, and the decision are captured at the moment it happens, not reconstructed afterward from application logs designed for debugging.
- Tamper-evident. Entries are hash-chained and append-only, so an alteration inside the chain fails verification rather than passing silently.
- Independently verifiable. This is the one that matters most. An auditor should be able to check the record on a machine you do not control, without taking your word for it and without taking the vendor's word for it either.
That last property is the whole argument. A platform cannot be the impartial witness to its own agents. Your AI vendor's logs are that vendor grading its own work. Your observability stack was built to help engineers debug prompts, not to convince a regulator. Your own application logs are your team certifying your team. Each of those is useful. None of them is independent — and independence is precisely what the question demands.
What this looks like in practice
Today, in most stacks, the honest answer to "prove what the agent did" is a query across application logs, a trace in an observability tool, a policy document in a GRC platform, and a person willing to attest that the three describe the same event. That works in a postmortem but it does not work at volume, and it does not work when the person asking has no reason to trust you.
The alternative is one record per action, written in the path of the call, carrying the action, the policy evaluated against it, and the decision that came back. Hash-chained. Exportable. Re-verifiable by whoever you hand it to, offline, using math rather than trust.
This is the artifact a governance program is supposed to produce and is the thing that turns "we cannot govern this at scale" into a solved problem rather than a permanent ceiling on what you are allowed to automate.
Autonomy is rate-limited by proof. Fix the proof and the ceiling moves.
If you are working through this, there is a short self-assessment that scores where you stand on seeing, recording, halting, and proving the AI already running in your business. Ten questions, three minutes, no email required.