Context Engineering for Engineers: The Context Assembly Pipeline

What Actually Gets Assembled

Anthropic’s own framing of context engineering names the concrete pieces a production agent has to manage: system instructions, tool definitions, message history, few-shot examples, and — for agents that use the Model Context Protocol — externally connected data sources. None of these are exotic; they’re just the ordinary inputs to any agent, and context engineering is the job of curating them rather than assuming “include everything” is a valid strategy.

“

[Good context engineering means] finding the smallest possible set of high-signal tokens that maximize the likelihood of some desired outcome.

Allocating a Context Budget

Because every category of context shares one fixed budget, giving more room to one category means less for another — the same trade-off you saw with the Introduction level’s backpack, just with real categories this time. Below is a small fixed task with three context categories, each capped at a maximum useful size and each with a different per-unit value. Try to find the highest-value allocation before checking the hint underneath.

Just-in-Time Retrieval

Rather than loading an entire knowledge base into context up front, agents built with a just-in-time approach keep only lightweight references — file paths, stored queries, web links — in context by default, and use tool calls to pull in the actual data only at the moment it’s needed. This keeps the resident context small and pushes the expensive, space-consuming step (fetching the real content) to exactly the turns where it’s relevant, instead of every turn.

Short-Term vs. Long-Term Memory

Message history is short-term: it lives only inside the current context window and disappears once the conversation ends. Retrieved data — documents, prior sessions, user preferences pulled from an external store — is long-term: it’s persisted outside the model entirely and only gets pulled into context on demand. Confusing the two is a common mistake: the context window itself is never a substitute for a real, persistent memory system.

Managing Context Over a Long Agent Run

For agents that run many turns in a row, three techniques keep context from overflowing without simply hoping the window is big enough:

• Compaction. Periodically summarizing older conversation turns into a shorter recap, freeing up window space while keeping the gist of what happened.
• Structured note-taking. The agent writes durable notes to external storage as it works, so critical facts survive even after the turns that produced them get compacted or dropped.
• Sub-agent architectures.A focused sub-agent handles one specific piece of work in its own isolated context, then returns only a condensed summary to the main agent — instead of every detail of that work polluting the main agent’s context.

Fun Fact

Anthropic’s own guidance treats context as what it calls “a precious, finite resource” — their framing is that this remains true even as context windows keep getting larger, because the problem was never really about running out of space. It was about the model’s attention degrading as low-value tokens pile up.

Test Yourself

Ready for production-scale evidence — a distinct failure mode from lost-in-the-middle, and a real multi-agent architecture built specifically to contain it? Continue to the Advanced level →