The complete guide to AI agent architecture

The problem

Agent architecture is often chosen by fashion. A team picks a framework, wires up a model and a handful of tools, and adds more agents when the first one struggles. A few months later it can have five agents passing messages to each other, no record of why any of them acted, credentials shared between them, and costs nobody can explain.

The model is rarely the problem. The hard work begins after the demo: grounding, evaluation, access, integration, latency, cost and reliable human escalation. Those are architecture decisions, and each is cheaper to make early than to retrofit.

Concept overview

An agent is a loop. It receives a goal and some context, decides what to do next, acts through a tool, observes the result, and repeats until the goal is met, it reaches a limit, or it hands over to a person.

Nine components make up that loop and its surroundings:

• Model — the reasoning engine. Replaceable, and it will be replaced.
• Instructions — the mission, the "never" rules, the escalation conditions; versioned like code.
• Tools — the actions the agent can take.
• Knowledge — approved content it answers from.
• Memory — what it carries between steps, sessions and interactions.
• State — where a long task has got to, so it can resume.
• Control loop — the logic that runs the steps, with limits on how many.
• Guardrails — the checks before and after each action, enforced outside the model.
• Observability — the trace of every step, call, cost and result.

When to use this

Not every problem needs an agent. If the steps are fixed and known in advance, build a workflow: it is cheaper, faster and easier to test. Use an agent when the work involves judgement over unstructured input and the steps vary from case to case. Anthropic's engineering guidance draws the same line — workflows follow predefined code paths, agents direct their own process — and the same advice: find the simplest solution, and add complexity only when it demonstrably improves outcomes. Most real systems are a mix: a workflow with agent steps where judgement is needed, and plain automation everywhere else.

Architecture

Agent systems grow in rungs. Climb one only when the rung below fails for a reason you can name.

The fifth rung is where the Citadel sits. Its layers — mission, wizard, agent, knowledge, integration, guardrail, observability, improvement — exist so that every agent, however many there are, has an owner, a boundary and a record.

Core components

Instructions

Write the system prompt as a contract: role, mission, definition of done, the "never" rules, when to escalate, how to cite. Keep it in version control and treat every change as a release that must pass the regression suite.

Tools

Narrow, typed and named for their purpose. Separate reading from changing. Make write tools safe to retry with idempotency keys, so a timeout does not create a duplicate. Return errors as information the agent can act on ("order not found"), not stack traces. Agent permissions and tool boundaries covers where limits are enforced.

Knowledge

Retrieved, permission-filtered, cited passages from owned sources; live numbers through tools, never from an index. Field Guide 017 covers the pipeline.

Memory

Separate the kinds:

• Working memory — the context for the current step. Keep it small: more context costs more, and models use long contexts unevenly.
• Session memory — the conversation or task so far.
• Long-term memory — facts kept across sessions: preferences, prior cases, decisions. Decide what may be written, who can read it, how long it lives, and how it is corrected.

Memory is not a database and not a knowledge base. Facts that must be accurate belong in a system of record; approved content belongs in the knowledge system; memory holds what the agent learned from its own work. How agent memory should work turns this into a design.

State: persistent and ephemeral agents

An ephemeral agent is created for one task and disappears with it: easy to reason about, test and scale. A persistent agent has a lasting identity, memory and schedule — it watches, follows up and owns a queue. It suits ongoing work and needs more care: lifecycle, memory hygiene, credential rotation and a way to pause it. Default to ephemeral (Persistent vs ephemeral agents).

Either way, a task that takes more than a few steps should checkpoint its state, so that a failure resumes rather than restarts — and does not repeat actions already taken.

Orchestration

When one agent becomes several, choose a pattern deliberately:

• Pipeline — fixed hand-offs: intake, then research, then draft, then review.
• Router — classify the request; send it to the right specialist.
• Orchestrator and workers — one agent plans and delegates; specialists execute and report back.
• Reviewer loop — one agent produces, another checks against criteria before anything leaves.

Messages between agents are data, not instructions. A specialist should not obey whatever another agent writes, any more than it would obey a document. Multi-agent orchestration covers the contracts and limits.

Guardrails and human checkpoints

Checks on input, a policy check before every tool call, checks on output. Consequential actions route to a person through an approval pattern (Field Guide 014). These live outside the model, where they cannot be talked out of.

Observability

Trace every run: the steps, the retrieved passages, each tool call with its parameters and result, tokens, cost and time. Without traces you cannot evaluate, debug or explain.

Implementation

The decisions, in the order they matter:

1. Mission — what the agent is for, and what it must never do.
2. Workflow or agent — automate the fixed steps; use an agent only for judgement.
3. Tools and permissions — the smallest set that does the job, under the agent's own identity.
4. Knowledge — owned sources, permission-aware retrieval, cite or abstain.
5. Memory and state — what persists and for how long; ephemeral by default; checkpoints for long tasks.
6. Human checkpoints — which actions need a person, and who.
7. Evaluation harness — scenarios and test tools before the first release.
8. Observability — traces and cost from day one.
9. More agents — only when one agent's scope has become too broad to test.
10. Operating layer — register, identities, approvals and records once several agents act in the same business.

The Governed Agent Blueprint turns decisions 1, 3, 6 and 7 into a one-page canvas.

Evaluation

Evaluate at three levels. Components: does retrieval find the right passage; does the agent choose the right tool with the right parameters? Trajectories: did it take a sensible path, or wander through ten calls to do the work of two? Outcomes: was the mission completed, within policy, at an acceptable cost? In a multi-agent system, add hand-offs: did the specialist receive what it needed, and did the orchestrator use what came back? Field Guide 028 covers the framework.

Security and governance

• One identity per agent, with its own credentials. In a multi-agent system each agent has only the permissions its own tools need — never the union of everyone's.
• Treat everything retrieved or received as data: documents, emails, web pages, tool results and other agents' messages. Indirect prompt injection — instructions hidden in content the agent reads — is a documented attack, not a hypothetical.
• Enforce limits outside the model, at the tool boundary. Excessive functionality, permissions and autonomy are what turn a wrong output into a damaging action.
• Record every action with the versions of model, instructions and tools in use.
• Give every agent a named owner. In the Guild, every deployed agent receives a role, an identity, approved tools, explicit limits, evaluation criteria and a named Human Wizard.

Cost considerations

Cost grows with the number of loop steps, the size of the context at each step, and the number of agents involved; a multi-agent design can multiply all three. The levers: cap steps and spend per task; keep context small; route simple steps — classification, extraction, routing — to smaller, cheaper models; cache stable results; and measure cost per completed mission, not cost per token. If adding an agent does not improve outcomes enough to pay for its tokens and its oversight, remove it.

Example implementation

Illustrative example, not a client engagement.

An operations team receives requests by email: access requests, invoice queries, supplier questions.

• Rung 2. One agent reads each email, looks up the requester and the relevant record, and drafts a reply for a person to send.
• Rung 3. It gains the team's procedures as a knowledge source, with citations, and a memory of open cases per requester.
• Rung 4. Invoice queries need finance tools the general agent should not hold. A router sends them to a finance specialist with its own identity and narrower permissions; access requests go to a person, always.
• Rung 5. With three agents acting, the team registers each one, gives each an owner, routes approvals through one queue, and keeps one activity record across all of them.

Each rung is added because the previous one failed for a reason someone could name.

Failure modes

Checklist

• The mission and the "never" rules are written before any architecture.
• Fixed steps are automated; agents handle judgement only.
• Tools are narrow, typed, idempotent and testable.
• Knowledge is owned, permission-aware and cited; live numbers come from tools.
• Memory types are separated, with rules for writing, reading, retention and correction.
• Agents are ephemeral unless the work genuinely continues.
• Long tasks checkpoint state and never repeat completed actions.
• Each agent has its own identity and only its own permissions.
• Messages from documents, tools and other agents are treated as data.
• Guardrails and approvals are enforced outside the model.
• Every run is traced, with cost.
• A new rung is added only for a reason someone can name.