- 01
Accountability cannot be delegated to a system — every automated action needs a named human owner recorded before go-live.
- 02
Authority limits belong in code, not in prompts; anything expressed only as an instruction is a preference, not a control.
- 03
The audit artefact of the agentic era is the trace: what the system knew, which policy version it applied, what it did and who could have stopped it.
The thesis
Enterprise AI governance was designed for a world of predictions and recommendations, where a human sat between the model and the consequence. That buffer is disappearing. When an agent issues a refund, adjusts a record, provisions access, sends a communication to a customer or closes a case, the consequence is real before anyone reviews it. Our view is that the governance question of 2027 is not whether models are accurate but who is accountable for what they do — and that most enterprises will discover their answer during an incident rather than before one. The organisations that get this right do something unglamorous: they write down, for every automated action, the human who owns the outcome, the limit beyond which the system must stop, and the record that will exist afterwards. That work is cheap in advance and extremely expensive to reconstruct after the fact, which is why it belongs on the 2026 agenda rather than the 2027 one.
Three controls that actually hold
First, named ownership. Every workflow that can take an action has a single accountable human recorded in the service catalogue, with the escalation path and the authority to suspend it. Diffuse ownership across a committee is equivalent to no ownership. Second, authority in code. The value threshold, the record types the agent may write, the customers it may contact and the actions requiring approval are enforced by the tool layer, which refuses anything outside the envelope regardless of what the model decides. Instructions in a prompt are guidance; the interface is the control. Third, the trace. For every action, a durable record of the inputs, the retrieved policy and its version, the reasoning path, the tool calls, the outcome and the identity under which it ran. These three are mutually reinforcing: named ownership without authority limits is unfair to the owner, authority limits without traces are unprovable, and traces without ownership are archaeology. Enterprises that implement all three find their regulatory conversations become straightforward.
Reversibility as the design principle
The most useful classification in agentic governance is not risk tier but reversibility. Actions that can be undone cheaply and invisibly — drafting, retrieving, tagging, scheduling internally — can be automated freely with monitoring after the fact. Actions that are reversible at a cost, such as a small refund or a record correction, can be automated with limits and sampling. Actions that are effectively irreversible — a payment beyond a threshold, an external communication to a regulator or a customer's counsel, a permanent deletion, a benefits or coverage determination — should keep a human decision point regardless of measured model accuracy, because the expected value calculation ignores tail consequences that the enterprise cannot absorb. Applied consistently, this principle makes governance faster rather than slower. Teams stop negotiating each workflow individually and know in advance which category they are in, and the approval burden concentrates where it matters instead of spreading thinly across everything.
The regulatory direction of travel
Requirements differ by jurisdiction but converge on the same demands: know where AI is used, document what it does, demonstrate that quality is measured, be able to explain a specific decision, and keep a human accountable for consequential outcomes. The EU AI Act phases obligations for general-purpose and high-risk systems across 2025 to 2027; sector regulators in financial services, healthcare and insurance are extending existing model-risk and consumer-duty regimes rather than writing new ones; and several jurisdictions are adding disclosure requirements when a customer is interacting with an automated system. Practically, an enterprise with an inventory of AI use cases, evaluation records, traces, named owners and a documented human-oversight design is prepared for nearly all of it. One is worth acting on now regardless of jurisdiction: maintain an AI system inventory. Almost every regime starts by asking what you have, and organisations that cannot answer that spend their first compliance cycle doing discovery instead of remediation.
Governance as a release gate, not a committee
The failure mode we see most often is governance implemented as a monthly review board. It creates a queue, teams route around it, and the board ends up approving things it cannot meaningfully assess. The alternative is governance encoded in the release path: a workflow cannot deploy unless it declares its accountable owner, passes its evaluation suite at a defined threshold, has its authority limits configured in the tool layer, emits traces, and states its reversibility class. All of that is checkable automatically. The committee's job then changes from approving deployments to setting the thresholds, reviewing incidents and auditing samples — work it is genuinely equipped to do. Enterprises that make this shift report the counter-intuitive result that governance accelerated delivery, because teams stopped waiting for meetings and started building against a known standard. This is the same transition security made when it moved from review boards to pipeline controls, and it takes about the same amount of political effort.
Implications by role
CIO: publish the AI system inventory and make it a condition of production access; nothing else in governance works without it. CISO: treat agent identity as first-class — agents need their own credentials, scoped entitlements and revocation paths, not a shared service account with broad rights. General Counsel: decide the reversibility classes and the human-oversight requirement per class in advance, so the business gets a rule rather than a case-by-case negotiation. Chief Risk Officer: extend model risk management to cover behavioural drift in production, because the risk profile of an agentic workflow changes when content, prompts or models change. Chief AI Officer: own the release gate and defend the evaluation thresholds against quiet weakening. CFO: fund the trace and evaluation infrastructure centrally, since it is the artefact every audit and regulator will ask for and no single workflow will pay for it.
What to put in place before 2027
Four artefacts cover most of the exposure, and all four can be produced in a quarter. A current inventory of every AI system in production with its owner, purpose, data and reversibility class. A written authority policy that states, per class, what agents may do unaided and what requires a human, enforced in the tool layer rather than in documentation. A trace standard applied uniformly, retained for a period agreed with legal, and queryable by case reference so a customer complaint can be answered in minutes. And an incident procedure specific to agent behaviour: how to suspend a workflow, how to identify every affected case, how to remediate and how to notify. Enterprises with these four handle their first agentic incident as an operational event. Enterprises without them handle it as a crisis, and the organisational response is usually to suspend the entire programme — which is how a single avoidable incident costs a year of progress.
- Accountability cannot be delegated to a system — every automated action needs a named human owner recorded before go-live.
- Authority limits belong in code, not in prompts; anything expressed only as an instruction is a preference, not a control.
- The audit artefact of the agentic era is the trace: what the system knew, which policy version it applied, what it did and who could have stopped it.
- Governance that runs as a release gate speeds delivery up; governance that runs as a review committee slows it down and gets bypassed.
- Regulatory readiness is largely a documentation discipline — the enterprises keeping traces and evaluation records already have most of what is required.
Questions leaders ask us
- Can accountability sit with the vendor?
- No. Contracts can allocate financial liability, but regulators, customers and boards hold the enterprise accountable for actions taken in its name. Vendor terms are a recovery mechanism, not a substitute for named internal ownership.
- Do authority limits in prompts count as controls?
- No. Anything expressed only as an instruction is a preference the system may not follow. Limits enforced by the tool layer, which refuses out-of-envelope requests, are the control.
- How long should traces be retained?
- Set the period with legal against the underlying business record — usually the same retention as the transaction the agent acted on. What matters more is that traces are queryable by case reference.
- Does this level of governance slow delivery down?
- Only if it runs as a committee. Encoded as automated release-gate checks, it speeds delivery up, because teams build against a known standard instead of waiting for approvals.
Sources
- [1] The EU AI Act phases obligations for general-purpose and high-risk AI systems across 2025–2027. Regulation (EU) 2024/1689 (Artificial Intelligence Act) — Official Journal of the European Union, 2024
- [2] Recognised AI risk management practice organises controls around govern, map, measure and manage functions. AI Risk Management Framework (AI RMF 1.0) — NIST, 2023