Multi-Agent Systems in Production: Breaking Down a Single Agent, Failure Modes, and Recovery

A Guideline: Breaking a Single Agent Into Orchestrator + Sub-Agents

Before splitting anything, work through these five steps in order:

1. 1. Identify genuinely independent sub-tasks

   List out the actual steps the single agent performs. Group them by dependency: does this step need the result of another step, or could it run on its own? Only steps with no real dependency on each other are candidates for separate sub-agents — forcing a dependent step into its own sub-agent just adds coordination overhead for no benefit.

2. 2. Define each sub-agent’s scope narrowly

   Each sub-agent should get one sub-task, a clear description of what “done” looks like for it, and only the tools it actually needs for that sub-task — not the full toolset the original single agent had. A narrow scope is what makes the sub-agent’s context stay small.

3. 3. Decide what a sub-agent reports back, before writing any code

   A sub-agent’s full working context should never reach the orchestrator — decide up front what condensed result it returns (a short summary, a specific structured value) so the orchestrator’s own context stays small too, the same boundary covered in the Intermediate level.

4. 4. Design the orchestrator’s combination logic explicitly

   Decide, in advance, exactly how the orchestrator will combine sub-agent results into a final answer — concatenation, a ranking, a synthesis step. This is real logic the orchestrator has to execute, not something that happens automatically just because multiple sub-agents ran.

5. 5. Plan failure and recovery before deploying

   Decide up front what happens when a specific sub-agent fails — not as an afterthought once it happens in production. The failure modes and recovery strategies below are exactly this: known-in-advance responses, not improvised ones.

Worked Example: Breaking Down a Single-Agent Research Assistant

Take a single agent whose job is: given a product idea, research three competitors and write a positioning summary. Written as one agent doing every step itself, in sequence:

# BEFORE: single agent, every step in sequence, one growing context

function run_single_agent(product_idea):
    context = new_context()
    context.add(product_idea)

    for competitor in ["Competitor A", "Competitor B", "Competitor C"]:
        pricing = search_web(competitor + " pricing")       # step depends on nothing
        reviews = search_web(competitor + " reviews")        # step depends on nothing
        context.add(competitor, pricing, reviews)            # but ALL results pile
                                                              # into one context

    positioning = generate_summary(context)                  # only this step actually
    return positioning                                       # needs everything gathered

# Problem: researching Competitor B doesn't need anything from
# Competitor A's research, but this code still does A, then B,
# then C, one after another — and every result sits in the same
# context, whether or not a later step needs it.

Applying the guideline: the three competitor-research steps are genuinely independent of each other (step 1 of the guideline), so each becomes its own narrowly-scoped sub-agent (step 2) that returns only a short positioning-relevant summary (step 3). Only the final summary step genuinely needs all three results gathered first — so it stays with the orchestrator, combining sub-agent outputs (step 4):

# AFTER: orchestrator fans out 3 independent sub-agents in parallel,
# then fans their results back in for the one genuinely dependent step

function research_subagent(competitor):
    pricing = search_web(competitor + " pricing")
    reviews = search_web(competitor + " reviews")
    # returns a CONDENSED result only — not the raw searches above
    return summarize(competitor, pricing, reviews)

function run_orchestrator(product_idea):
    competitors = ["Competitor A", "Competitor B", "Competitor C"]

    # fan-out: all 3 sub-agents run at the same time, independently
    results = run_in_parallel([
        research_subagent(c) for c in competitors
    ])

    # fan-in: gather results back together once all 3 finish
    orchestrator_context = new_context()
    orchestrator_context.add(product_idea, results)  # only 3 short
                                                      # summaries here,
                                                      # not the raw
                                                      # searches behind them

    positioning = generate_summary(orchestrator_context)
    return positioning

Two concrete things changed, not just the code’s shape. The three research steps now genuinely run at the same time instead of one after another (real latency win, since none of them ever depended on each other). And the orchestrator’s own context holds three short summaries instead of every raw search result from all three competitors — the same boundary shown in the Intermediate level’s diagram, now actually reflected in the code’s structure, not just described in the abstract.

Failure Modes in Production

The orchestrator-worker code above is the happy path — three sub-agents, no conflicts, nothing fails. Real deployments have to plan for all four of these on their own:

• Coordination overhead

  Every sub-agent the orchestrator spawns costs real setup and combination work — assigning the sub-task, waiting on the result, merging it in. Split a task into too many overly narrow sub-agents, and the coordination cost can exceed what a single agent would have spent on the whole task itself.

• Duplicate work

  Two sub-agents scoped ambiguously enough can end up covering the same ground — both researching an overlapping piece of the task — wasting tokens and sub-agent time on work neither orchestrator nor either sub-agent noticed was redundant.

• Conflicting sub-agent outputs

  Independent sub-agents researching related things can legitimately disagree — one reports a competitor’s price as $49, another (from a different source) reports $59. Nothing in the architecture resolves this automatically; the orchestrator has to have explicit logic for handling contradictions, or it silently picks whichever result arrived last.

• Error propagation across agents

  A wrong or incomplete result from one sub-agent flows into the orchestrator’s final combination step exactly like a correct one would, unless the orchestrator validates results before combining them. A single bad sub-agent output can quietly corrupt an otherwise-correct final answer.

Recovery Modes: What Happens After a Sub-Agent Fails

Noticing a failure mode occurred is only half the job — the orchestrator also needs a chosen response, decided before deployment, not improvised in the moment. Try each recovery strategy on the same failed sub-agent below, and notice that none of them is universally correct — each one trades something away:

Most production systems don’t pick just one recovery mode for every failure — they escalate through a few in order. A common real pattern: retry once (cheap, catches transient failures), then degrade gracefully if the retry also fails (the user still gets a usable, honestly-labeled answer), and only fail the whole task outright if the missing piece makes the entire result meaningless without it. Laid out as a flow, cheapest option first:

Using Failure Modes and Recovery Modes Together

Failure modes and recovery modes aren’t two separate topics — a failure mode is the thing you have to detect, and a recovery mode is what you do once you’ve detected it. Not every failure mode pairs well with every recovery mode, and picking the wrong pairing is itself a common production mistake:

The mismatch to actually watch for: retrying a sub-agent doesn’t fix conflicting outputs — both sub-agents already succeeded, so retrying just produces a third opinion, not a resolution. And degrading gracefully doesn’t fix coordination overhead — the task still costs the same in setup and combination work whether or not one piece is dropped from the final answer. Matching the recovery mode to what actually caused the failure is what makes recovery logic effective, rather than a generic retry-everything reflex.

The Real Cost: Multiple Agents Means Multiple LLM Calls

Every sub-agent spawned is a separate set of model calls, not a free parallel thread. Splitting a task across 3 sub-agents plus an orchestrator, in the worked example above, means at minimum 4 separate agent runs instead of 1 — and Anthropic’s own measurement of this exact trade-off (cited in the Introduction level) found their multi-agent research system used roughly 15× more tokens than a single agent to get a 90.2% improvement on their internal evaluation. That ratio is the real question to answer before splitting any specific task: is the accuracy or latency gain actually worth paying for that many more tokens, for this task, specifically?

Fun Fact

“No recovery — fail the whole task” sounds like the worst option in the simulator above, but it’s sometimes the most honest one. A task where a missing piece genuinely invalidates the rest of the answer (like a financial calculation missing one required input) is better served by a clear failure than a plausible-looking partial answer the user might mistake for complete.

Test Yourself