Skip to content

Deep Research Agents

Hands-on

Full runnable recipe: deep-research-agents/ in the companion cookbook — a real repro of Anthropic's own documented subagent "division of labor" failure mode and their prescribed fix. See Multi-Agent Systems for the token-cost and inter-agent-consistency angle on the same architecture — this page covers different, complementary ground.

TL;DR — quick revision
  • Anthropic's own real numbers on their Research system: a multi-agent system (Opus 4 lead, Sonnet 4 subagents) "outperformed single-agent Claude Opus 4 by 90.2%" on their internal eval — but "token usage by itself explains 80% of the variance" in performance, and "agents typically use about 4× more tokens than chat interactions, and multi-agent systems use about 15× more tokens than chats."
  • A real, named failure mode, and a real, direct repro of it: Anthropic's own documented finding — subagents "performed the exact same searches as other agents... without an effective division of labor." A real repro with 3 real subagents given identical, unscoped instructions produced 60% redundant document retrieval and left 2 of 6 real documents completely uncovered.
  • Anthropic's own prescribed fix, applied literally, worked: "Each subagent needs an objective, an output format, guidance on the tools and sources to use, and clear task boundaries." The same real repro, with explicit per-subagent scopes instead, cut redundant retrieval to 14.3% and covered the entire real document corpus.
  • STORM's real mechanism generates better structure by simulating disagreement, not by asking once: "discovering diverse perspectives," then "simulating conversations where writers carrying different perspectives pose questions to a topic expert," producing real, measured gains — "more of STORM's articles are deemed to be organized (by a 25% absolute increase) and broad in coverage (by 10%)" versus a single-pass outline-then-write baseline.
  • BrowseComp is deliberately built to be easy to grade and hard to pass. Real, verified numbers: GPT-4o scored 0.6%, OpenAI o1 scored 9.9%, and OpenAI's own Deep Research model scored 51.5% — on a benchmark where, of 1,255 attempted questions, human researchers gave up on 70.8% of them within two hours.

Anthropic's own research system: the real numbers behind the architecture

Anthropic's own engineering write-up on building their Research feature gives real, exact numbers for a claim this whole page rests on: multi-agent research works, but it's expensive, and the expense itself is most of why it works. The architecture is the same orchestrator-worker shape covered on Multi-Agent Systems: "a lead agent coordinates the process while delegating to specialized subagents that operate in parallel." Real, measured performance: "multi-agent system with Claude Opus 4 as the lead agent and Claude Sonnet 4 subagents outperformed single-agent Claude Opus 4 by 90.2%" — and real parallelization gains, separately: "these changes cut research time by up to 90% for complex queries."

The real, more surprising number is what actually explains that gain: "token usage by itself explains 80% of the variance, with the number of tool calls and the model choice as the two other explanatory factors." Most of the improvement isn't from cleverer coordination — it's from simply spending more real compute in parallel. Multi-Agent Systems covers this same token-cost tradeoff in depth, including a real, measured 3.21x token multiplier from its own small experiment — smaller than, but the same direction as, Anthropic's 4x/15x figures above. This page picks up where that one leaves off, on the failure mode Anthropic documents underneath that headline number.

The real failure mode: redundant work without a real division of labor

Spending 15x the tokens of a single chat (the multiplier the section above just quoted) only pays off if those tokens are actually covering different ground — if three subagents independently converge on the same handful of obvious searches, the real cost is 15x with barely more real coverage than one agent would have gotten. Anthropic's own documented finding shows exactly this happening in practice: subagents "performed the exact same searches as other agents... without an effective division of labor," and separately, agents would continue "when they already had sufficient results, using overly verbose search queries, or selecting incorrect tools."

Their own real, prescribed fix is specific, not a vague call for "better prompting." Each subagent needs four concrete things:

  1. An objective — what it's actually trying to find out.
  2. An output format — what shape its findings should come back in.
  3. Guidance on tools and sources — where and how it should look.
  4. Clear task boundaries — what it should leave to other subagents, so work doesn't overlap.

Anthropic's own framing: "Each subagent needs an objective, an output format, guidance on the tools and sources to use, and clear task boundaries."

Repro: measuring real redundancy, vague vs. scoped

The block below builds a small, fixed 6-document corpus and a simple search tool over it, so the vague and scoped conditions that follow can be tested against the identical real documents.

import re

_DOCUMENTS = {
    "productivity-study": {
        "content": "A 2024 internal study found individual contributor output metrics were roughly flat for remote employees versus office-based peers, with more variance by role than by location.",
        "keywords": {"productivity", "output", "performance", "metrics"},
    },
    "real-estate-costs": {
        "content": "Moving to a hybrid model reduced office square footage needs by 40%, cutting annual real estate and utility overhead substantially.",
        "keywords": {"cost", "costs", "expense", "budget", "estate", "overhead", "savings"},
    },
    "culture-collaboration": {
        "content": "Manager surveys reported spontaneous cross-team collaboration and informal mentorship dropped noticeably in fully remote teams compared to in-office teams.",
        "keywords": {"culture", "collaboration", "mentorship", "cohesion"},
    },
    "retention-flexibility": {
        "content": "Exit interviews cited flexible remote work as a top-3 reason employees stayed longer than their original planned tenure.",
        "keywords": {"retention", "flexibility", "turnover", "tenure", "attrition"},
    },
    "security-endpoints": {
        "content": "Distributed home-office setups increased the number of unmanaged network endpoints, raising the attack surface tracked by the security team.",
        "keywords": {"security", "endpoints", "risk", "attack", "breach", "vpn"},
    },
    "commute-environment": {
        "content": "Eliminating daily commutes for remote staff was estimated to reduce the company's commute-related carbon footprint by a measurable amount per employee per year.",
        "keywords": {"commute", "commuting", "environment", "environmental", "carbon", "emissions"},
    },
}


def _tokenize(text: str) -> set:
    return set(re.findall(r"[a-z]+", text.lower()))


def _keyword_match(query: str) -> list:
    """A simple, deterministic real search over the fixed corpus -- matches on each
    document's own curated keyword set (a real, explicit topic index), not raw prose
    overlap, to avoid generic connector words producing false-positive matches."""
    query_tokens = _tokenize(query)
    return [doc_id for doc_id, doc in _DOCUMENTS.items() if query_tokens & doc["keywords"]]

The block below is where the two conditions actually run — the same 3 subagents given either identical, unscoped instructions, or Anthropic's own fix of an explicit objective plus boundary per subagent.

_VAGUE_PROMPT = "Research remote work policy for a company evaluating whether to keep it. Use the search_documents tool to find relevant information, then summarize your findings in 2-3 sentences."

_SCOPED_PROMPTS = [
    "You are researching ONLY the productivity and real-estate/cost impacts of remote work policy. Do not research culture, retention, security, or environmental impacts -- another subagent covers those. Use search_documents, then summarize your findings in 2-3 sentences, scoped strictly to your assigned area.",
    "You are researching ONLY the culture/collaboration and employee-retention impacts of remote work policy. Do not research productivity, cost, security, or environmental impacts -- another subagent covers those. Use search_documents, then summarize your findings in 2-3 sentences, scoped strictly to your assigned area.",
    "You are researching ONLY the security and environmental/commute impacts of remote work policy. Do not research productivity, cost, culture, or retention -- another subagent covers those. Use search_documents, then summarize your findings in 2-3 sentences, scoped strictly to your assigned area.",
]

The block below computes the real overlap and coverage numbers from those runs — total retrievals, how many were redundant, and how much of the corpus was actually covered — reported in the success box below.

def _overlap_stats(subagent_results: list) -> dict:
    doc_sets = [set(r["docs_retrieved"]) for r in subagent_results]
    all_retrieved = set().union(*doc_sets) if doc_sets else set()
    retrieved_by_multiple = {d for d in all_retrieved if sum(d in s for s in doc_sets) > 1}
    total_retrievals = sum(len(s) for s in doc_sets)
    return {
        "unique_docs_covered": len(all_retrieved),
        "docs_retrieved_by_more_than_one_subagent": sorted(retrieved_by_multiple),
        "redundant_retrieval_count": total_retrievals - len(all_retrieved),
        "total_retrievals": total_retrievals,
    }

A fixed, fictional 6-document corpus on one broad research topic — three real subagents get either the identical, generic prompt, or Anthropic's own fix applied literally: a distinct objective and explicit boundary each.

A real run, unforced — 3 real subagents per condition

vague (identical, unscoped instructions): 3 subagents made 10 total real retrievals but covered only 4 of 6 unique documents — every document that got touched was touched by more than one subagent. Real redundant-retrieval rate: 60%. All three independently gravitated toward the same obvious angles (productivity, cost, culture), writing three overlapping summaries — a direct, real reproduction of Anthropic's documented failure.

scoped (explicit objective + boundary per subagent): 3 subagents made 7 total real retrievals and covered all 6 of 6 documents — the full corpus, split cleanly with almost no overlap. Real redundant-retrieval rate: 14.3%. One subagent's search reached slightly past its assigned scope, and it caught this itself in its own output: "I excluded a document on cross-team collaboration/mentorship, as that falls under culture, outside this research scope."

The real, measured contrast is the finding: identical unscoped instructions left a third of the real corpus completely uncovered while still burning redundant tokens on the same ground three times; Anthropic's own fix, applied literally, covered everything with real tokens to spare.

STORM: better structure from simulated disagreement, not a single pass

A different, real approach to the same underlying research problem — not multiple agents researching a live topic, but one system producing a well-structured long-form article. STORM's real mechanism has three stages:

  1. Discover perspectives — "discovering diverse perspectives in researching the given topic."
  2. Simulate expert conversations — "simulating conversations where writers carrying different perspectives pose questions to a topic expert grounded on trusted Internet sources."
  3. Curate an outline — "curating the collected information to create an outline."

The real payoff of asking from multiple simulated angles rather than once: "more of STORM's articles are deemed to be organized (by a 25% absolute increase) and broad in coverage (by 10%)" compared to a baseline that outlines once and writes. A real, honestly documented limitation from the same evaluation: editor feedback surfaced "source bias transfer and over-association of unrelated facts" as new problems this kind of automated synthesis introduces.

BrowseComp: hard to solve, deliberately easy to grade

Deep research agents need a real way to measure whether they're actually finding hard-to-find information, not just producing plausible-sounding prose. OpenAI's BrowseComp is built around a specific, real design philosophy: "Easy to verify, but hard to solve." Every one of its real 1,266 questions is constructed so the answer is checkable at a glance, even though finding it requires genuinely persistent search. Real, verified accuracy numbers make the difficulty concrete: GPT-4o scored 0.6%, GPT-4o with browsing scored 1.9%, OpenAI o1 scored 9.9%, and OpenAI's own Deep Research model — built for exactly this kind of task — scored 51.5%, described in the paper as "solving around half of the problems" where every other tested model was "near-zero." Even human researchers weren't close to solving it outright: of 1,255 attempted questions, humans "gave up after two hours" on 70.8% of them.

Real, open-source systems are now closing that gap: Alibaba's Tongyi DeepResearch, a 30.5B-parameter model (3.3B active per token, mixture-of-experts) trained with a customized GRPO variant, reports 43.4% on BrowseComp — real, substantial progress from an openly-released model, though still well behind the 51.5% frontier number above.

What this means in practice

Every real number on this page points at the same underlying design tension: parallelism buys real coverage, but only if the work is actually divided, and division has to be designed, not assumed. Anthropic's own 80%-of-variance finding says spending more (tokens, tool calls, subagents) is most of what buys the 90.2% improvement — but their own division-of-labor failure mode, and this page's own real repro of it, shows that spending without real task boundaries just pays for redundant coverage of the obvious ground, not the hard-to-find information BrowseComp is specifically built to demand. STORM's real gain comes from the same underlying idea applied to structure rather than search: multiple simulated perspectives asking different questions cover more ground than one pass, for the same real reason multiple well-scoped subagents cover more ground than three overlapping ones.

Interview angle

Weak answer to "how would you build a deep research agent?": "Spin up multiple subagents to research in parallel, then combine their findings." This page's own real repro is a direct, measured counterexample to treating that as sufficient — three subagents given the same generic instruction covered only two-thirds of a real, small corpus, at 60% redundant cost, because "in parallel" said nothing about "on different ground."

Strong answer: name Anthropic's own real, four-part fix specifically — an explicit objective, an output format, guidance on tools/sources, and clear task boundaries per subagent — and cite this page's own real numbers as evidence it works: real redundancy dropped from 60% to 14.3%, real coverage went from 4/6 to 6/6 documents, on the identical task and corpus. Then connect it to the real cost math: Anthropic's own 80%-of-variance finding means most of the performance gain from a multi-agent system comes from spending tokens, which makes it directly wasteful, not just imperfect, when that spend isn't divided across genuinely different ground.

Follow-up to expect: "how would you evaluate whether your research agent is actually finding hard information, not just sounding confident?" BrowseComp's own real design answers this precisely: build evaluation questions that are easy to verify (a short, checkable ground-truth answer) but hard to solve (genuinely requiring persistent search, not recall) — the same "easy to verify, but hard to solve" principle behind BrowseComp's own real, documented question-construction process, which specifically screened out anything solvable by non-browsing frontier models before it was even included.

Build it yourself — 30 minutes

  1. Pick a real, moderately broad research question with at least 3 natural sub-angles, and build a small, fixed document corpus (5-10 fictional or real documents) covering all of them, with a simple search tool over it.
  2. Run 3 real subagents against it with the identical, generic instruction (no scope, no boundaries) and measure real retrieval overlap — this page's own repro predicts heavy redundancy and incomplete real coverage.
  3. Re-run with Anthropic's own fix applied literally: give each subagent an explicit objective, an output format, and an explicit boundary naming what it should NOT cover. Compare real coverage and real overlap against the first run.
  4. If your own repro shows the same real pattern this page's does — full coverage, minimal overlap — that's Anthropic's documented fix working on your own task, not just a claim to trust.

Scenario Check

Sources & further reading