Choosing and Building the Right One in Production

The Real Cost: LLM Call Count

Call count is a real, countable fact about each approach’s structure — not an estimated benchmark score. Pick an approach below to see how predictable (or not) its cost actually is:

Getting the Choice Wrong: Over- and Under-Engineering

• Under-engineering: chatbot for a multi-step task

  A support bot that only replies in one turn, asked to “check my order status and process a refund if it’s late,” can only describe what a human should do — it can’t actually look up the order or issue the refund. The user leaves with a description of a solution, not a solved problem.

• Over-engineering: full agent for a fixed sequence

  A request that always follows the exact same 3 steps — classify the ticket, look up the account, send the templated reply — doesn’t need an agent deciding its own path each time. A full agent here adds a variable, harder-to-predict call count and new failure modes (like a runaway loop) for a task a fixed 3-step workflow already handles reliably, at a fixed, predictable cost.

The Same Task, Three Ways, in Code

The Problem

A customer support inbox receives a ticket like: “I was charged twice for my subscription this month — can you fix this?” Handling it properly means three concrete things have to happen, in this order: figure out what kind of ticket this is (billing, technical, or general), look up the actual account behind it (not a guess — real account status, real billing history), and send a replythat’s grounded in that real data rather than generic advice. “Done” specifically means: the right category was identified, the real account was actually queried, and the customer received a reply that reflects their real situation — not just a plausible-sounding response.

This is a deliberately good task for comparing all three approaches, because it has a fixed, well-known shape (classify → look up → reply) that never changes — which is exactly the property a fixed workflow is built to exploit, and exactly the property that makes a full agent’s extra flexibility unnecessary here. Watch, across all three versions below, for the same two failure points: does it ever touch real account data, and does its cost stay predictable from one ticket to the next?

The Code

Here’s the exact same task, in code, at all three points on the spectrum:

1. Chatbot (1 call, no real action)

def handle_ticket_chatbot(ticket_text):
    # A single call. No tools, no lookup, no real action taken.
    return llm_call(
        f"A customer wrote: '{ticket_text}'. Suggest how support "
        f"should typically respond to this kind of ticket."
    )

# handle_ticket_chatbot("I was charged twice for my subscription this month")
# -> "This sounds like a billing issue — the support team should check
#     the customer's payment history for a duplicate charge and process
#     a refund if one is confirmed."
# Advice about what SHOULD happen — never actually looked at this
# customer's real billing history, never confirmed anything, never
# refunded anything.

2. Workflow (fixed 3-call routing pattern)

def handle_ticket_workflow(ticket_text, account_id):
    # Call 1: routing — classify the ticket into a fixed set of categories
    category = llm_call(
        f"Classify this ticket into exactly one category: "
        f"billing, technical, or general. Ticket: '{ticket_text}'"
    )

    # Real action: look up the account (a real tool, not a suggestion)
    account = lookup_account(account_id)

    # Call 2: generate a reply using the real category and real account data
    reply = llm_call(
        f"Write a reply for a '{category}' ticket. "
        f"Account status: {account['status']}. Ticket: '{ticket_text}'"
    )

    # Call 3: a fixed evaluator pass, always run, checking tone before sending
    approved = llm_call(f"Does this reply sound professional? Reply: '{reply}'")

    return reply if approved == 'yes' else escalate_to_human(ticket_text)

# handle_ticket_workflow("I was charged twice for my subscription this
#                         month", account_id="acct_4471")
# Call 1 -> category = "billing"
# lookup_account("acct_4471") -> {'status': 'active', 'last_charge':
#                                  '$29.99 x2 on Jul 24', 'duplicate': True}
# Call 2 -> reply = "You're right — we see two $29.99 charges on Jul 24.
#                     We've refunded the duplicate; it'll post in 3-5 days."
# Call 3 -> approved = "yes"
# -> returns the reply above. Always exactly 3 LLM calls, in this exact
#    order, every single run — and this time, grounded in the real
#    duplicate charge actually found in the account.

3. Full agent (variable call count, LLM decides its own path)

MAX_ITERATIONS = 8

def handle_ticket_agent(ticket_text, account_id):
    state = {'ticket': ticket_text, 'account_id': account_id}
    for i in range(MAX_ITERATIONS):
        # The LLM itself decides the next action — not a fixed step order
        action = decide_next_action(state)  # returns a tool call or 'done'
        if action['tool'] is None:
            return action['final_reply']
        state[action['tool']] = call_tool(action['tool'], **action['args'])
    raise RuntimeError('Exceeded max_iterations without resolving the ticket')

# handle_ticket_agent("I was charged twice for my subscription this
#                      month", account_id="acct_4471")
# Iteration 1 -> decides to call lookup_account -> finds the duplicate charge
# Iteration 2 -> decides it also needs check_refund_eligibility (a step the
#                fixed workflow never has, because this account happens to
#                be inside a promotional period with different refund rules)
# Iteration 3 -> decides it now has enough to reply -> 'done'
# -> resolved in 3 calls this run. A different account with no special
#    rules might resolve in 2. A messier ticket needing a second lookup
#    might take 6. Call count is NOT fixed — it depends on what the agent
#    actually discovers along the way, which the workflow's fixed 3-step
#    path has no way to react to at all.

Run against the exact same double-charge ticket, the three versions diverge exactly where the Problem section said to watch: the chatbot never touches this customer’s real billing history at all — it only guesses that a duplicate charge is likely. The workflow always makes exactly 3 calls, in the same fixed order, and this time actually confirms and refunds the real duplicate charge found in the account. The agent also resolves it correctly, but for a completely unrelated reason its call count isn’t predictable in advance: this particular account turned out to be in a promotional period with different refund rules, so the agent needed an extra step the fixed workflow doesn’t even know exists. A different account, with nothing unusual about it, would have resolved in fewer steps — the same ticket text, a different real call count.

A Guideline: When to Actually Move Up the Spectrum

1. 1. Start with a single, well-optimized LLM call

   Before adding any workflow or agent complexity, confirm a single call — with good retrieval and in-context examples — genuinely can’t handle the task. Anthropic’s own guidance is explicit: for most applications, this is already enough.

2. 2. Move to a fixed workflow only when the task has a genuinely fixed shape

   If the task always follows the same sequence of steps — classify, then look up, then reply — pick the workflow pattern (from the Intermediate level) that matches that shape. This gets you real multi-step capability while keeping cost and latency fixed and predictable.

3. 3. Move to a full agent only when the sequence genuinely can’t be fixed in advance

   If the number or order of steps depends on what’s discovered along the way — debugging an unknown failure, open-ended research — no fixed workflow pattern will fit. This is the only case that actually requires an agent, not a preference for “more capable-sounding” architecture.

4. 4. Measure before and after, don’t assume

   Anthropic’s own framework ties every complexity increase to demonstrated improvement, not intuition. Before shipping a workflow or agent, confirm — with a real evaluation, not a guess — that it actually outperforms the simpler approach on the task at hand.

Fun Fact

The workflow version above still checks its own reply with a third, fixed evaluator call before sending it — the same evaluator-optimizer pattern from the Intermediate level, nested inside a routing pattern. Real production systems frequently combine two or more of the five named patterns in a single task, rather than picking exactly one and stopping there.

Test Yourself