Stop Blaming Your Security Team for AI Agent Breaches

Post-incident reviews of AI agent breaches routinely ask the wrong question. Organisations ask who approved this agent's access — a diligence question suited to conventional systems, where permissions are fixed and enumerable.

AI agents break that assumption. They compose their actions at runtime, so the correct permission set cannot be known in advance. This is not a failure of process discipline. It is a structural limitation of asking a person to predict behaviour that has not yet occurred.

The instinct to blame, and why it misdiagnoses the problem

When an AI agent causes damage — a data leak, an unauthorised action, a manipulated decision — the first question inside most organisations is: who approved this agent's access?

It is the natural question to ask. It is also the wrong one.

In the large majority of cases, the answer is not negligence. It is that the person configuring the agent's permissions was asked to do something that is not actually achievable: predict, in advance, the exact set of actions an autonomous system will decide to take once deployed.

Why this differs from every other access decision

HUMAN EMPLOYEE
  Determined by   fixed role, known responsibilities
  Predictability  high — reviewable and stable

CONVENTIONAL APPLICATION
  Determined by   fixed at build time, visible in code
  Predictability  high — the binary defines the ceiling

AI AGENT
  Determined by   composed at runtime, from task and data encountered
  Predictability  low — a property of behaviour, not of code

An AI agent's permission requirement is not a property of its code. It is a property of what it actually does, which cannot be known until it is already doing it. The same agent, given identical starting access, may call three tools on one run and twelve on the next, depending entirely on the task and the data it encounters along the way.

Faced with this, organisations have effectively two options. Restrict the agent almost entirely, which defeats the purpose of deploying it. Or grant broad standing access and monitor for problems after the fact, which most teams choose — not out of carelessness, but because it is the only workable option available with current tooling.

Choosing the second is a rational response to an unsolved tooling gap. It is not a diligence failure, and treating it as one leads organisations toward fixes that do not address the actual cause.

Why the standard post-incident response does not close the gap

The typical organisational response to an agent-related incident is to tighten the manual approval process: additional sign-offs, more documentation, longer review cycles.

This does not solve the underlying problem. It makes the guess more expensive to produce. It does not make it more accurate. The permission set is still being authored by a person under the same fundamental uncertainty as before; the process is simply slower.

The correct question is not "who approved this access."

The correct question is: what is the minimum set of permissions this specific task actually requires, and how would we determine that without asking a person to predict it in advance?

Toward a different category of solution

That reframing points to a different approach entirely: deriving the policy from observed behaviour, rather than authoring it under uncertainty.

The mechanism, at a conceptual level. Observe what a defined task class genuinely requires over a bounded, controlled observation period. Treat that observed evidence — not a person's best guess — as the basis for the permission boundary. Update the boundary as behaviour is re-observed, rather than fixing it once and reviewing it only after an incident.

This shifts the burden from prediction to measurement — a category of problem organisations are far better equipped to solve reliably.

Implications for risk and governance functions

This distinction changes how an organisation should evaluate its own exposure. If the current basis for agent permissions is "a person wrote a policy based on their best understanding of what the agent might need," that policy is only ever as accurate as that person's ability to predict runtime behaviour — a ceiling defined by the nature of the problem, not by the competence of the person doing the work.

The fix is not a better analyst or a stricter checklist.

The fix is a different source for the policy altogether.