Skip to content

Multi-Agent Systems

Hands-on

Full runnable recipe: multi-agent-systems/ in the companion cookbook — Anthropic's and Cognition's opposing claims about multi-agent systems, tested directly on real toy tasks with real measured numbers.

TL;DR — quick revision
  • A multi-agent system is not the same thing as orchestrator-workers. Workflow Patterns' orchestrator-workers is a fixed topology — Anthropic classifies it as a workflow. A multi-agent system is what Anthropic calls an agent — a lead agent that delegates to subagents that are themselves autonomous, operating in parallel with their own judgment, their own tool calls, and their own context windows.
  • A real measured run put a real number on Anthropic's token-cost claim: single agent answering three independent questions, 1 call, 1,048 tokens; a lead agent dispatching three subagents plus a synthesis call, 4 calls, 3,369 tokens — a real 3.21x multiplier, the same direction as (if smaller than) Anthropic's own reported 4x/15x figures.
  • A real test of Cognition's inter-agent consistency risk did not reproduce the failure — twice. Two subagents, each blind to the other's output, independently inventing a shared fact (a trial length) converged on the same value both times, even after the prompt was redesigned specifically to remove an obvious reason they'd default to the same answer. Reported as an honest negative result, with the real, disclosed reason why the repro's own design likely couldn't have shown the failure Cognition describes.
  • MAST's real taxonomy: 14 distinct failure modes across 3 categories — system design issues, inter-agent misalignment, task verification — built from 1,600+ annotated traces across 7 frameworks, and the paper's own stated finding is that multi-agent systems' "performance gains on popular benchmarks are often minimal."
  • "When to use one agent" has a real, non-hand-wavy answer: genuinely independent, breadth-first sub-tasks are where the token cost buys something real (Anthropic's own 90.2% improvement, largely attributable to spending more tokens); tightly-coupled tasks needing shared context between steps are where a single agent avoids a coordination problem it would otherwise have to solve by hand.

Multi-agent systems vs. orchestrator-workers

This is the distinction most likely to get muddled in an interview: two things that look almost identical on a diagram — a lead box, worker boxes underneath it — aren't the same underlying thing. In one, each worker is a fixed function call: a pre-built role the developer wired in ahead of time, filling in judgment only at the specific points the code already anticipated. In the other, each worker is itself a full, autonomous agent — deciding its own actions, with its own tool access, its own reasoning loop, its own context window. Anthropic's own writing draws this precisely.

In the orchestrator-workers workflow pattern, "a central LLM dynamically breaks down tasks, delegates them to worker LLMs, and synthesizes their results" — but the workers are single calls executing a fixed role, the same way every pattern on the Workflow Patterns page is a fixed topology with an LLM filling in judgment at specific, pre-built decision points.

A multi-agent system, in the sense Anthropic uses for their own Research product, is different in kind: "a multi-agent architecture with an orchestrator-worker pattern, where a lead agent coordinates the process while delegating to specialized subagents that operate in parallel" — each subagent deciding its own actions rather than executing one pre-scripted call. That difference is exactly why multi-agent systems carry the specific risks this page tests (token cost, coordination failure) that an orchestrator-workers workflow, by design, doesn't — a workflow's workers can't disagree with each other in a way the code didn't already anticipate, because they aren't making open-ended decisions to begin with.

The debate: Anthropic vs. Cognition

Two influential, opposite-leaning posts, published within weeks of each other in mid-2025. Anthropic's engineering team, describing their own production Research system: multi-agent with Opus 4 as lead and Sonnet 4 subagents "outperformed single-agent Claude Opus 4 by 90.2%" on an internal breadth-first research eval (identifying board members across S&P 500 IT companies) — but with a cost, stated just as plainly: "agents typically use about 4x more tokens than chat interactions, and multi-agent systems use about 15x more tokens than chats," and on a separate benchmark (BrowseComp), "token usage by itself explains 80% of the variance" in performance — meaning much of multi-agent's advantage there is bought with tokens, not architecture.

Cognition's post argues close to the opposite, from direct experience building coding agents: "running multiple agents in collaboration only results in fragile systems. The decision-making ends up being too dispersed and context isn't able to be shared thoroughly enough between the agents." Their concrete example: two subagents build a Flappy Bird clone, one given the sub-task of the background, the other the bird — the background subagent produces something that "looks like Super Mario Bros," while the bird "doesn't look like a game asset and it moves nothing like the one in Flappy Bird" — a coherent single vision fractured into two locally-reasonable, globally-inconsistent pieces, left for a final agent to somehow reconcile. Their stated principle: "actions carry implicit decisions, and conflicting decisions carry bad results" — because each subagent only sees the sliver of context it was handed, not the full trace of decisions the other subagent made along the way.

Both are right about their own evidence, and the difference is task shape, not which company is correct — which is exactly what the two real experiments below test.

Real experiment 1: token economics

The code below runs the identical three independent questions through both architectures — one call handling all three, versus a lead agent dispatching one subagent per question plus a synthesis call — so the token counts and ratio below are measured directly, not estimated.

def _call(prompt, client, model, system=None, max_tokens=600):
    # client.messages.create(...) -- see agent_loop.py for the full call shape.
    ...


QUESTIONS = [
    "Explain, in one paragraph, how a Bloom filter achieves space-efficient set-membership testing.",
    "Explain, in one paragraph, how consistent hashing reduces cache invalidation when nodes are added or removed.",
    "Explain, in one paragraph, how a skip list achieves O(log n) search without tree-style rebalancing.",
]


def single_agent_research(client, model) -> dict:
    prompt = "Answer each of these three independent questions, one paragraph each, clearly labeled:\n\n" + "\n".join(
        f"{i+1}. {q}" for i, q in enumerate(QUESTIONS)
    )
    text, tokens = _call(prompt, client, model, max_tokens=900)
    return {"answer": text, "calls": 1, "total_tokens": tokens}


def multi_agent_research(client, model) -> dict:
    """A lead agent dispatches one subagent per question -- each subagent gets
    its own system framing and answers only its question, blind to the
    other two -- then a synthesis call assembles the combined answer."""
    subagent_system = "You are a research subagent. Answer only the question you are given, thoroughly, in one paragraph."
    subagent_results = []
    total_tokens = 0
    for q in QUESTIONS:
        text, tokens = _call(q, client, model, system=subagent_system, max_tokens=400)
        subagent_results.append(text)
        total_tokens += tokens

    synthesis_prompt = "Combine these three subagent answers into one clearly labeled response, one paragraph each:\n\n" + "\n\n".join(
        f"Subagent {i+1} answer: {r}" for i, r in enumerate(subagent_results)
    )
    final_text, synth_tokens = _call(synthesis_prompt, client, model, max_tokens=900)
    total_tokens += synth_tokens

    return {"answer": final_text, "calls": len(QUESTIONS) + 1, "total_tokens": total_tokens}

A real run — three genuinely independent questions, two architectures, real token counts

Input, identical three questions for both conditions: explain (1) how a Bloom filter achieves space-efficient set-membership testing, (2) how consistent hashing reduces cache invalidation, (3) how a skip list achieves O(log n) search without tree-style rebalancing — genuinely independent of each other, the shape Anthropic's own post argues favors multi-agent.

Single-agent (Claude Sonnet 5, one call given all three questions): 1 call, 1,048 total tokens.

Multi-agent (a lead agent dispatching one subagent per question — each subagent blind to the other two — then a synthesis call combining all three): 4 calls, 3,369 total tokens.

Ratio: 3.21x. Smaller than Anthropic's own reported 4x (agent vs. chat) or 15x (multi-agent vs. chat) — those are measured on real production workloads against a plain-chat baseline, not against a single-agent-doing-everything baseline on one small toy task — but the same direction, on a real number this page computed itself rather than borrowed. The mechanism is visible in the design itself: each subagent call repeats system framing overhead ("You are a research subagent...") that a single combined call only pays once, and the synthesis call has to re-read all three subagent outputs in full before producing the final answer — tokens single-agent's one pass never had to spend at all.

Real experiment 2: consistency risk

The code below has two subagents independently invent a shared fact — a fictional app's free-trial length — each blind to the other's output, to test whether Cognition's "conflicting decisions" risk actually shows up when two agents have to agree on the same underspecified detail.

import re

# A refund paragraph mentions two day-counts (the refund window itself, and
# the trial length it's contrasted against), and a day-count near "trial"
# isn't always in a tight fixed window -- of every "N day(s)" match, pick
# the one with the smallest character distance to any "trial" mention,
# rather than assuming fixed proximity.
_DAY_NUMBER_RE = re.compile(r"(\d+)[\s-]*days?", re.I)
_TRIAL_RE = re.compile(r"trial", re.I)


def _extract_trial_length(text: str):
    trial_positions = [m.start() for m in _TRIAL_RE.finditer(text)]
    if not trial_positions:
        return None
    best = None
    best_distance = None
    for match in _DAY_NUMBER_RE.finditer(text):
        distance = min(abs(match.start() - p) for p in trial_positions)
        if best_distance is None or distance < best_distance:
            best, best_distance = int(match.group(1)), distance
    return best


# Deliberately asks for an unusual day-count, not a common default like 7,
# 14, or 30 -- those have strong enough cultural conventions that two
# independent guesses would likely converge by default, which wouldn't
# test the actual risk of two blind subagents inventing different values.
PRICING_INSTRUCTION = (
    "Write one short FAQ paragraph for a fictional app called Nimbus, under the heading 'Pricing'. "
    "Mention that new users get a free trial before being charged, and state how many days the trial lasts "
    "(there is no official policy yet -- pick a specific, slightly unusual number of days, not a common "
    "default like 7, 14, or 30)."
)
REFUND_INSTRUCTION = (
    "Write one short FAQ paragraph for a fictional app called Nimbus, under the heading 'Refunds'. "
    "Explain the refund window, and mention that no refund is needed during the free trial period -- "
    "reference how many days the trial lasts (there is no official policy yet -- pick a specific, slightly "
    "unusual number of days, not a common default like 7, 14, or 30)."
)


def single_agent_faq(client, model) -> dict:
    """One agent writes both sections in the same pass -- whatever trial
    length it invents, it invents once and can reuse."""
    prompt = f"{PRICING_INSTRUCTION}\n\n{REFUND_INSTRUCTION}\n\nWrite both sections."
    text, tokens = _call(prompt, client, model, max_tokens=500)
    sections = text.split("Refund")
    pricing_section = sections[0]
    refund_section = "Refund" + sections[1] if len(sections) > 1 else text
    pricing_trial = _extract_trial_length(pricing_section)
    refund_trial = _extract_trial_length(refund_section)
    return {
        "full_text": text, "tokens": tokens,
        "pricing_trial_days": pricing_trial, "refund_trial_days": refund_trial,
        "consistent": pricing_trial is not None and pricing_trial == refund_trial,
    }


def multi_agent_faq(client, model) -> dict:
    """Two subagents, each blind to the other's output -- exactly the
    scenario Cognition's post argues breaks down: 'actions carry implicit
    decisions, and conflicting decisions carry bad results.'"""
    pricing_text, pricing_tokens = _call(PRICING_INSTRUCTION, client, model, max_tokens=250)
    refund_text, refund_tokens = _call(REFUND_INSTRUCTION, client, model, max_tokens=250)
    pricing_trial = _extract_trial_length(pricing_text)
    refund_trial = _extract_trial_length(refund_text)
    return {
        "pricing_text": pricing_text, "refund_text": refund_text,
        "tokens": pricing_tokens + refund_tokens,
        "pricing_trial_days": pricing_trial, "refund_trial_days": refund_trial,
        "consistent": pricing_trial is not None and pricing_trial == refund_trial,
    }

A real run — an honest negative result, with the real reason disclosed

Input — two FAQ sections for a fictional app, Nimbus: a Pricing section stating a free-trial length in days, and a Refunds section that must reference that same trial length when explaining no refund is needed during it. Deliberately underspecified — "there is no official policy yet, pick a specific, slightly unusual number of days, not a common default like 7, 14, or 30" — designed to force each writer to genuinely invent a value rather than recall one everyone already agrees on.

Single-agent (one call, both sections written in the same pass): picked 19 days in Pricing, referenced 19 days in Refunds. Consistent — trivially, since it's one generation with the value fixed the moment it was first written.

Multi-agent (two subagents, each given only its own section's instruction, each blind to the other's output — exactly Cognition's "conflicting decisions" scenario): the Pricing subagent picked 11 days; the Refunds subagent, asked separately and with no visibility into the Pricing subagent's choice, also landed on 11 days. Consistent.

This didn't reproduce Cognition's failure — on this run, or on an earlier attempt before the "avoid common defaults" instruction was added (which converged on 14 days both times, before the fix). The honest, disclosed reason is a real limitation of this specific repro, not evidence the underlying risk is overstated: Cognition's own example is about open creative interpretation — what a background or a game character looks like, with genuinely high variance and no shared convention to fall back on. This repro tested convergence on a single scalar fact, and two independent runs suggest the same model, even instructed to be unusual, tends to agree with itself on a number more than the stylistic, high-variance choices Cognition's example turns on. A harder, more faithful repro of the consistency risk would test creative or structural divergence, not a number — real future work this page doesn't claim to have already done.

MAST: how multi-agent systems actually fail

Beyond one company's coding-agent anecdote or another's research-system win, MAST (Cemri et al.) is the closest thing to a systematic answer: a taxonomy built from rigorous analysis of 150 real multi-agent traces (kappa = 0.88 inter-annotator agreement), then validated against 1,600+ annotated traces across 7 popular frameworks. Fourteen distinct failure modes, clustered into three categories:

  1. System design issues — problems baked in before the system ever runs: unclear role or task specification, agents stepping on each other's responsibilities, missing conversation history that a later step actually needed.
  2. Inter-agent misalignment — the category Cognition's Flappy Bird example lives in: agents talking past each other, making incompatible assumptions, or failing to actually incorporate what another agent already established (real echoes of "actions carry implicit decisions, and conflicting decisions carry bad results" from a completely independent research effort).
  3. Task verification — nobody checks the final output is actually right; a subtly wrong result from one stage propagates all the way to the end because no step's job was to catch it.

The paper's headline finding is worth stating plainly, because it's easy to only remember the wins: "despite enthusiasm for Multi-Agent LLM Systems (MAS), their performance gains on popular benchmarks are often minimal." That's not a contradiction of Anthropic's 90.2% — it's a reminder that the win is real for the task shape it was measured on (breadth-first, parallelizable, worth the token spend), and MAST's broader survey across many frameworks and many task types finds that gain doesn't generalize automatically to every multi-agent deployment.

When to use one agent

Both companies' own evidence, plus MAST's broader survey, point to the same practical answer, stated as precisely as the evidence allows: reach for multi-agent when a task genuinely decomposes into independent pieces that don't need to see each other's intermediate reasoning to succeed — Anthropic's board-member research task is the clean case, and this page's own token-economics experiment shows that architecture costs real tokens (3.21x, on our own small task) in exchange for that parallelism. Stay with a single agent when steps are coupled — when a later step's correctness depends on the specific reasoning path an earlier step took, not just its final output — because that's exactly the condition under which Cognition's "conflicting implicit decisions" risk applies, and this page's own consistency-risk experiment, even though it didn't reproduce a failure, tested exactly the shape of task (shared, underspecified fact needing agreement across pieces) where that risk structurally exists. MAST's task-verification category adds a third, orthogonal axis that applies regardless of the choice: whichever architecture is used, something has to actually check the final output is right, because neither a fixed workflow nor an autonomous multi-agent system does that for free.

Interview angle

Weak answer to "would you use a multi-agent system for this task": "Multi-agent is more powerful, so yes, for anything complex enough to benefit from parallelism." This treats multi-agent as a strictly-better upgrade rather than a real trade-off, and it can't explain either half of this page's own real evidence — the 3.21x token cost that's real regardless of task, or the specific shape of task (tightly-coupled, shared-context-dependent) where Cognition's risk structurally applies even when this page's own repro happened not to trigger it.

Strong answer: the question to ask is whether the task's sub-parts are genuinely independent — can each one be solved correctly using only its own slice of context, with no need to see what another piece decided? If yes, multi-agent's parallelism and specialization are worth the measured token premium, the way Anthropic's board-member research task was. If no — if a later piece's correctness depends on a decision an earlier piece made that isn't fully captured in its final output — a single agent with the whole task in one context window avoids a coordination problem that a multi-agent system would otherwise have to solve explicitly (shared state, full-trace sharing, or an explicit verification step), which is real, non-trivial engineering Cognition's post argues most teams underestimate.

Follow-up to expect: "your own consistency-risk experiment didn't find a failure — doesn't that undercut the argument for being cautious?" No — it's a scoped, honest result, not a general one. The real finding is that a single scalar fact, chosen by the same model twice independently, tends to converge — which says something specific and narrow, not that inter-agent inconsistency isn't a real risk. MAST's 1,600+-trace survey and Cognition's own creative-task example are the broader evidence that the risk is real; this page's own repro tested one narrow instance of it and, honestly, wasn't positioned to reproduce the failure — a fact worth stating precisely rather than either overclaiming a disproof or hiding an inconvenient result.

Build it yourself — 30 minutes

  1. Pick a task with three or more genuinely independent sub-parts — something where each part's correct answer doesn't depend on knowing the others. Run it as one agent, one call, and record the real token count from your API response's usage field.
  2. Run the same task as a lead agent dispatching one subagent call per sub-part, plus a synthesis call. Record the real token count the same way. Compare the real ratio — don't estimate it.
  3. Design a second task where two pieces must agree on one shared, underspecified fact. Run it as one agent (both pieces in one pass) and as two subagents, each blind to the other's output. Extract the fact from each output programmatically and check agreement — don't just skim and assume it's fine.
  4. If your consistency check doesn't find a mismatch, don't discard it — ask why, honestly, the way this page did. Was the shared fact a scalar with a strong convention, or something genuinely high-variance like Cognition's creative example? The answer changes what the result actually tells you.

Scenario Check

Sources & further reading