Memory, Planning and Multi-Agent Systems

The context window is the agent's entire “working memory”, and it is finite. The longer the task, the sharper the question: what to keep in context, what to move outside, and when to run several agents instead of one. These are the main topics of agent engineering after the basic loop.

Intuition

Imagine you are solving a big task, but you can only keep notes on one whiteboard of a fixed size. At first there is plenty of room. Then the board fills up with logs and drafts, the important stuff drowns in noise, and new notes push out old ones. What does a sensible person do? Erases what's done, leaving a short summary; keeps a separate notebook for the important things; and hands big independent chunks to a colleague with their own clean board. Exactly these three techniques — compaction, external memory and subagents — make up an agent's memory management.

How it works

Context is working memory. Everything goes into the context window: the system prompt, the plan, tool calls and their results. The window is finite, and worse — quality degrades even before it overflows: the model makes worse use of information from the middle of a long context (the “lost in the middle” effect). Practical tactics:

• Compaction / summarization. Old messages — especially verbose tool results — are compressed into a short summary. A history of “100 tokens of logs” becomes “tests were failing because of an import, fixed” in 10 tokens.
• Notes in external files (scratchpad). The agent writes what matters to a file outside the context: findings, decisions, todos. The file survives any compaction and overflow — this is long-term memory.
• Selective loading. Instead of dragging the whole history along, the agent pulls in the needed fragments of its notes as required — essentially RAG over its own history.

Planning. For a complex task it is useful to materialize the plan as an artifact — a todo list in the context or in a file. It pins down the decomposition, survives compaction and lets you notice drift: if the agent has long been doing something other than what's in the plan, something went wrong.

Multi-agent systems. An orchestrator hands out subtasks to subagents. The main benefit: each subagent has its own clean context — it can read twenty files, try five hypotheses, and all that noise stays with it; only a short result returns to the orchestrator. Hence the patterns: parallel fan-out over independent subtasks, vertical roles (executor, reviewer, critic). The cost: information loss in handoff (the subagent doesn't know the whole dialogue), more tokens in total, harder debugging. That's why for a simple linear task that fits into one context, multi-agent is over-engineering.

What to keep in context and what to move out

A good heuristic is three tiers. Always in the window: the system prompt, the current plan, the latest observations. Compressed: older history — as a summary after compaction. Outside: bulky artifacts (logs, dumps, intermediate documents) — in files, from which the agent loads what it needs. A symptom of broken tiers: the agent “forgets” agreements from the start of the session — meaning something important fell under compaction instead of living in the plan or the scratchpad.

When to go multi-agent

Signals “for”: the subtasks are independent and can run in parallel (fan-out: “check these 5 modules”); a subtask generates a lot of noise that isn't needed upstream (a deep search through a codebase); you need an independent perspective (a reviewer who hasn't seen the author's reasoning and so isn't infected by its mistakes). Signals “against”: the task is short and linear; the subtasks are tightly coupled through shared state — then passing context between agents eats the whole gain and adds “broken telephone” errors.

All interview questions in this section: Agents and Tools — interview questions