The most important rule in agent security fits in one sentence: the model is not a security boundary.
Anything you tell an agent not to do in its instructions is a request. The model will usually honour it. But models make mistakes, misread situations, and can be manipulated by text planted in an email, a web page or a document they were asked to read — the risk OWASP lists first for LLM applications, as prompt injection. A permission is different. It is something the surrounding system enforces whatever the model decides. The architecture of an agent is largely the question of where those enforcements sit.
This note sets out the layers, what each is for, and how to test that they hold. Sentinel's dispatch 5 permissions an agent should never have covers which powers should never be granted at all; this note is about how every other permission is built.
Five places a boundary can live
| Layer | The question it answers | Example |
|---|---|---|
| 1. Tool catalogue | Does this capability exist for this agent at all? | The support agent has no payments tool |
| 2. Tool design | What can this tool possibly do? | issue_refund(order_id, amount), not "call any API" |
| 3. Policy check | Is this call, by this agent, for this person, allowed now? | Refunds above a limit need approval |
| 4. Credential | What can the tool's access do if everything above fails? | A key scoped to refunds on one account |
| 5. Downstream system | Does the target system itself agree? | The billing system enforces its own roles |
Each layer catches what the one before it misses. Together they mean that a confused or manipulated model meets several independent refusals before anything happens in the world.
1. The tool catalogue: absent beats forbidden
The safest permission is a tool that does not exist. An agent cannot misuse a capability it was never given, and it cannot be talked into using it. Give each agent the smallest set of tools its mission needs, and treat every addition as a change that needs a reason and a review. OWASP's guidance on excessive agency names this first: minimise the extensions an agent can reach.
2. Tool design: narrow, typed, explicit
A tool's signature is a boundary. A narrow tool with typed parameters, enumerated options and server-side validation can only do what it says. A generic tool — run a query, call a URL, execute a script — can do anything its credential allows, which means it has no boundary of its own. Separate tools that read from tools that change things. Block sensitive fields explicitly: an "update supplier" tool that can also change bank details is two tools pretending to be one.
3. The policy check: decisions outside the model
Every tool call should pass through a check that the model cannot influence. It takes the facts of the call — which agent, acting for whom, which action, on which resource, with which parameters, under which mission and autonomy level — and returns one of four answers: allow, allow within limits, require approval, or deny.
| Tool | Action | Scope | Limits | Above the limit |
|---|---|---|---|---|
lookup_order | Read | Orders of the customer in the conversation | — | — |
issue_refund | Change | Orders of that customer, in the last 90 days | Per refund, per day and per customer, set by the finance owner | Approval by a support lead |
send_email | Change | The customer in the conversation only | Per hour | Deny |
update_address | Change | That customer's delivery address | Once per order | Approval |
Limits deserve as much care as rules. Count cumulatively, not per call: an agent that may refund up to a limit should not be able to issue three refunds just under it. Add rates, volumes, time windows and a cost ceiling per task. The check belongs in code, close to the tool — in the Citadel's terms, the guardrail layer of permissions, policies and escalation.
4. The credential: what happens if everything else fails
Each tool runs with its own credential, scoped to exactly what that tool needs, held outside the model and never shown to it. The agent acts under its own identity, never a person's, so every action is attributable and access can be revoked without locking a colleague out. Prefer short-lived credentials issued per task. Then ask the uncomfortable question: if the policy check had a bug, what could this credential do? The answer is your real blast radius.
5. The downstream system: complete mediation
Finally, the system being acted on should enforce its own permissions, as it would for any user. OWASP calls this complete mediation: authorisation is enforced in the downstream system rather than relying on the model's judgement. It is the layer that still holds when the agent, the gateway and the credential are all wrong in the same direction.
Acting on behalf of someone
Agents usually act for a person: the customer in the conversation, the employee who asked. The effective permission must then be the intersection of what the agent may do and what that person may do. An agent with access to every customer's records must not become a way for one customer to read another's. This is the classic confused deputy problem — a privileged component tricked into using its authority for someone who lacks it — and it is avoided by passing the requesting person's identity through to the tools and checking it there, which OWASP describes as executing in the user's context.
Approvals are a permission state
An approval is not a pop-up; it is one of the outcomes of the policy check. Design it so that it means something:
- Bound to the exact action. What the person approves is the specific call with its parameters, and only that call can then run. Approval for one refund cannot be spent on another.
- Carrying evidence. The request shows what the agent wants to do, why, and what it looked at.
- Expiring. An approval left unanswered for a week should not quietly execute next month.
- Recorded. Who approved, when, and on what evidence, in the activity record.
Reading is not harmless
Read access looks safe and often is not. An agent that can read sensitive data, is exposed to content an attacker can influence, and can send information out — by email, a web request, even a link it renders — can be manipulated into exfiltrating that data. Simon Willison calls this combination the lethal trifecta. The defence is architectural: avoid giving one agent all three, or put a hard boundary on the third, such as sending only to the customer in the conversation and only through templates.
Sentinel note — When I review an agent, I do not start from its instructions. I list its tools, then read what each tool's credential can actually do, then look for the pairs: sensitive reads next to outbound channels. The instructions tell me what the agent intends. The credentials tell me what it can do.
Test the boundary, not the model
Boundaries are tested like any other capability, with scenarios built to break them:
- requests just above a limit, and one large action split into several small ones;
- instructions planted in documents, emails and tool results;
- a request to act for a different person than the one in the conversation;
- tool errors and timeouts mid-action, to check that retries do not duplicate;
- a scripted "agent" that ignores its instructions and calls tools directly, to prove that the policy check and credentials stop it without the model's cooperation.
The last test matters most, because it tests the boundary rather than the model's good behaviour. In the Guild's Evaluation Lab, Sentinel owns the boundaries — security, permissions, policy adherence, risk and escalation rules — and policy adherence is one of the dimensions every agent is scored on.
Keep a permission register
For each agent, keep one current record: its tools, each tool's credential and scope, the limits, the autonomy level per action, the owner, and the date of the next review. Changes to the register are approved by a person — an agent may propose a permission change but never approves its own — and every permission can be revoked in one step. This is what lets a buyer, an auditor or a board ask "what can this agent do?" and receive an answer in minutes.
The Security & Trust page puts the commitment simply: every deployed agent receives a role, an identity, approved tools, explicit limits, evaluation criteria and a named Human Wizard. For agents that reach into cloud accounts, the same layers apply at the infrastructure level; Cross-account AWS agent deployments shows how. And the blueprint for deciding tools and permissions in the first place is in The Governed Agent Blueprint.
