Agent Architecture Fundamentals: Memory, Planning, State, and Multi-Agent Patterns

Strip the branding off any agent framework and you find the same four subsystems: memory (what the agent knows right now and what it can recall later), planning (how it decides the next step), durable state (what survives a crash, a deploy, or a human approval gate), and coordination (how multiple agents divide work). Architect those four deliberately and the framework question becomes a packaging decision.

That ordering matters for buyers. Most enterprise agent evaluations start with a framework bake-off and inherit whatever memory, planning, and state semantics the winner ships. This guide inverts that: it lays out the architectural building blocks and their research lineage, so you can write requirements a platform team can test — and so a vendor demo can't substitute charisma for a checkpointing story.

1. The boundary that decides everything else: workflow or agent?

Before any memory or planning decision, settle whether you need an agent at all. Anthropic's "Building Effective Agents" essay draws the line cleanly: workflows are "systems where LLMs and tools are orchestrated through predefined code paths," while agents are "systems where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks."^[3] The same essay is blunt about which to reach for first: workflows offer predictability and consistency for well-defined tasks; agents earn their cost only "where it's difficult or impossible to predict the required number of steps, and where you can't hardcode a fixed path."^[3]

Classic ML orchestration sits at the far deterministic end of that spectrum. AWS describes SageMaker Pipelines as "a purpose-built workflow orchestration service to automate machine learning (ML) development,"^[4] where a pipeline is "a series of interconnected steps in directed acyclic graph (DAG)" whose structure is fixed by data dependencies between steps.^[5] A DAG cannot decide to take a different path at runtime — and for retraining, evaluation, and deployment jobs, that inflexibility is precisely the feature: reproducibility, lineage, and auditability come free.

The practical architecture is rarely either/or. A common enterprise shape is a deterministic outer pipeline — scheduled, audited, compliance-friendly — that invokes a bounded agentic loop for the one step that genuinely needs judgment (an investigation, a synthesis, a triage). That containment keeps the agent's variability from contaminating the parts of the system your auditors care about.

2. Memory: three tiers, three different engineering problems

Working memory is the context window — and it is finite

An agent's short-term memory is its context window: the prompt, conversation history, and tool results currently in view. Anthropic's guidance on context engineering treats this as the binding constraint of agent design — context "must be treated as a finite resource with diminishing marginal returns," with a model's "attention budget" depleted by every token and performance degrading as context grows (a phenomenon the essay calls context rot).^[6] The design consequence: an agent that dumps every tool result into history is not being thorough, it is spending down the budget that its later reasoning needs.

The mitigation is active curation. Compaction summarizes a conversation approaching the window limit and reinitializes with the compressed version — done well, it "distills the contents of a context window in a high-fidelity manner, enabling the agent to continue with minimal performance degradation."^[6] Clearing stale tool results is the lightweight version of the same idea. Ask any agent-platform vendor to show you their compaction strategy; "we have a big context window" is not one.

Long-term memory lives outside the model

Long-term memory — user preferences, accumulated project knowledge, learned facts — belongs in external storage the agent reads and writes deliberately: files, databases, vector stores. The clearest architectural statement of this pattern is the MemGPT paper, which proposes "virtual context management, a technique drawing inspiration from hierarchical memory systems in traditional operating systems," in which the system "intelligently manages different memory tiers in order to effectively provide extended context within the LLM's limited context window."^[7] The same idea now ships as product capability: Anthropic describes agents using a memory tool to maintain notes outside the context window, so they can build knowledge over time and reference previous work without keeping everything in context.^[6]

For an enterprise, long-term memory is where the governance load concentrates. Anything an agent persists about a customer or an employee is a record: it needs retention policy, access control, residency treatment, and a deletion path. Evaluate memory backends the way you evaluate any system of record — because that is what they become.

Episodic memory: the record of what the agent did

The third tier is episodic — structured traces of past attempts and events, retrievable by relevance rather than replayed wholesale. The research lineage here is direct. The Generative Agents work built agents on an architecture that extends the language model "to store a complete record of the agent's experiences using natural language, synthesize those memories over time into higher-level reflections, and retrieve them dynamically to plan behavior."^[8] Reflexion made episodic memory operational for task performance: agents "verbally reflect on task feedback signals, then maintain their own reflective text in an episodic memory buffer to induce better decision-making in subsequent trials."^[2] Episodic memory is also your audit trail — the same traces that help the agent improve are what let a reviewer reconstruct why it acted.

3. Planning: three loops with real research lineage

Planning approaches cluster into three patterns, each anchored in a citable paper rather than framework marketing. They differ on one axis: when does the agent commit to a plan, and when does it revise?

ReAct (Yao et al., 2022) interleaves thought and action step by step. The paper's core claim: generating "both reasoning traces and task-specific actions in an interleaved manner" creates synergy — "reasoning traces help the model induce, track, and update action plans as well as handle exceptions, while actions allow it to interface with external sources, such as knowledge bases or environments, to gather additional information."^[9] On interactive benchmarks, ReAct "outperforms imitation and reinforcement learning methods by an absolute success rate of 34% and 10% respectively" (ALFWorld and WebShop), prompted with only one or two in-context examples.^[9] ReAct is the default loop inside most modern tool-using agents: adaptive, simple, and legible — every step leaves a reasoning trace an operator can read.

Plan-and-execute separates the two phases: draft an explicit multi-step plan first, then carry it out, replanning only when a step fails. The research root is plan-then-solve decomposition — Plan-and-Solve prompting proposes "first, devising a plan to divide the entire task into smaller subtasks, and then carrying out the subtasks according to the plan," specifically to address the missing-step errors that plague purely step-by-step reasoning.^[10] Operationally, the upfront plan is a governance artifact: it can be reviewed, cost-estimated, or approved by a human before any action executes — which is why regulated workflows gravitate here. The tradeoff is brittleness: a bad plan fails in bulk, so the replanning path is not optional.

Reflexion (Shinn et al., 2023) adds a third loop around either of the above: try, critique, retry. The framework reinforces "language agents not by updating weights, but instead through linguistic feedback" — the agent reflects on failure signals in text and stores those reflections in its episodic buffer for the next attempt.^[2] The headline result — 91% pass@1 on HumanEval against GPT-4's 80%^[2] — comes with a condition buyers should notice: self-reflection pays off where an automatic feedback signal exists (tests pass or fail, a query returns or errors). Where success is subjective, the critique loop mostly adds tokens and latency.

4. Durable state: the unglamorous layer that decides production-readiness

A demo agent lives for one request. A production agent handles work that spans failures, deploys, rate limits, and — most importantly — human approval gates that may take days. That requires state to be a first-class, persisted object, not variables in a process. The framework ecosystem has converged on a name for this: durable execution. LangGraph, the most widely referenced implementation, describes it as the ability to "build agents that persist through failures and can run for extended periods, automatically resuming from exactly where they left off," alongside human-in-the-loop inspection and modification of agent state at any point during execution.^[11]

Mechanically, durable execution means checkpointing: after each step, the runtime serializes the agent's state — conversation, plan position, pending tool calls — to a persistent store keyed by a session or thread identifier. Resumption loads the latest checkpoint and continues. Three design decisions follow. Granularity: checkpoint per step gives clean resumption but adds write load; checkpoint per milestone is cheaper but replays work after a crash. Storage: a durable database for checkpoints, with a fast cache only as an optimization layer in front of it — never as the system of record. Validation: a resumed agent should verify the world still matches its checkpoint (the ticket it was processing may have been closed by a human in the meantime) before acting on stale intent.

Durable state is also your compliance surface. Checkpoints plus episodic traces give you replayable history: what the agent knew, what it decided, who approved it. Teams that treat state persistence as an afterthought end up rebuilding it under audit pressure — at which point it is a migration, not a design choice.

5. Multi-agent patterns: orchestrator-worker first, peers rarely

When one agent's context or focus isn't enough, the dominant production pattern is orchestrator-workers: in Anthropic's formulation, "a central LLM dynamically breaks down tasks, delegates them to worker LLMs, and synthesizes their results" — distinct from simple parallelization because the subtasks aren't predefined but chosen by the orchestrator.^[3] Anthropic's production research system is the best-documented instance: a lead agent plans, spawns parallel subagents to explore independent aspects, and compiles their condensed findings; the multi-agent configuration beat single-agent Claude Opus 4 by 90.2% on the internal research eval, and parallel tool calling "cut research time by up to 90% for complex queries."^[1]

The architectural reason this works is context isolation, not headcount. Each subagent burns its own context window on exploration and returns only a distilled summary — Anthropic's context-engineering guidance describes subagents returning condensed summaries, often 1,000 to 2,000 tokens, to the coordinating agent — so the orchestrator's window holds conclusions, not raw search transcripts.^[6] Multi-agent design is context-window economics before it is org design.

Peer patterns — agents conversing as equals rather than reporting to an orchestrator — have a serious research and framework base: Microsoft's AutoGen is "an open-source framework that allows developers to build LLM applications via multiple agents that can converse with each other to accomplish tasks," with agents that are "customizable, conversable, and can operate in various modes that employ combinations of LLMs, human inputs, and tools."^[12] Fully decentralized swarm designs push further: no coordinator, local autonomy, indirect coordination through a shared environment or blackboard-style workspace. The honest enterprise read is that decentralization wins in narrow conditions — when work partitions cleanly along fault, geography, or data-residency boundaries and no global consensus is needed — and loses everywhere else, because debugging emergent behavior across peers is harder than reading one orchestrator's delegation log. Start hierarchical; decentralize only the seams that prove independent.

Whatever the topology, agents need a standard way to reach tools and data. The Model Context Protocol is the current center of gravity: "an open standard that enables developers to build secure, two-way connections between their data sources and AI-powered tools," replacing fragmented per-source integrations with a single protocol.^[13] For architecture purposes, MCP's significance is decoupling: tool servers become infrastructure shared across agents and frameworks, so switching frameworks stops meaning rebuilding every integration.

6. Honest objections

"We don't need any of this — a good prompt and a workflow engine cover our use cases." Often correct, and the primary sources agree: Anthropic's own guidance is to start with simple prompts, optimize them with comprehensive evaluation, and add multi-step agentic systems only when simpler solutions fall short.^[3] If your task list is retraining pipelines, document routing, and templated generation, a DAG plus retries is the better architecture — cheaper, auditable by construction, and boring in the way production systems should be.^[4] The four subsystems in this guide are for the residue of tasks where fixed paths genuinely fail.

"The benchmark numbers won't transfer to our workload." Also fair. ReAct's absolute gains were measured on ALFWorld and WebShop^[9]; Reflexion's 91% is HumanEval^[2]; Anthropic's 90.2% is an internal research eval.^[1] None of these is your claims-processing queue. Treat the numbers as evidence that the mechanisms work — grounding, reflection, parallel decomposition — not as forecasts of your lift. The transferable lesson from Anthropic's eval work is the method: build your own evaluation before scaling the architecture.

"Frameworks will abstract all of this away — why architect it ourselves?" Partly true: durable execution, memory tools, and orchestration primitives are increasingly framework features rather than custom builds.^[11] But abstraction moves the decision, it doesn't remove it. Someone still chooses checkpoint granularity, memory retention policy, and coordination topology — and if nobody on your side does, the framework's defaults become your compliance posture. Understanding the fundamentals is what makes you a customer who can evaluate, rather than inherit, those defaults.

7. The read: four decisions to make on paper first

For a CIO or platform lead, this pillar compresses into four writable requirements. Memory: demand a stated context-curation strategy (compaction, tool-result clearing) and treat any persistent memory store as a governed system of record. Planning: match the loop to the task — ReAct for adaptive tool use, plan-and-execute where a human must approve before actions run, reflection loops only where feedback is automatic. State: make durable execution and demonstrated mid-step crash recovery a hard acceptance criterion for anything long-running. Coordination: default to a single well-contexted agent; adopt orchestrator-workers when parallel exploration provably pays for its token multiple; treat peer decentralization as a special case, not a default. Frameworks then become the last step: pick the one that implements the architecture you specified — a decision covered in the companion framework comparison below.