Planning and Decomposition¶

Hands-on

Full runnable recipe: planning-and-decomposition/ in the companion cookbook — real plan-and-execute repros: a replanning trigger, a granularity tradeoff that produced a real, reproducible failure, and a minimal verifier-in-the-loop check.

TL;DR — quick revision

Plan-and-execute separates the "what should happen" decision from the "make it happen" work: a planner produces a multi-step plan up front, and an executor carries it out — real, cited benefits over a single ReAct-style loop include speed ("the larger agent doesn't need to be consulted after each action"), cost ("sub-tasks... can be made to smaller, domain-specific models"), and completion quality ("forcing the planner to explicitly 'think through' all the steps required").
A real replanning repro was a clean, honest negative: told either to execute its plan without second-guessing, or explicitly to revise on a broken assumption, Sonnet 5 produced the identical, correct outcome both times when a planned step's premise (a repo has a git tag) turned out false. Worth reporting plainly — the assumption that a rigid "don't second-guess" instruction would cause a real failure didn't hold up.
A real granularity repro found the opposite — a genuine, reproducible failure, confirmed twice: the same task, decomposed too coarse or well-sized, completed correctly both times (3 real tool calls, a correct changelog). Decomposed over-granular, the model spent its entire token budget writing out sub-steps and never called a single tool — a real, measurable cost of over-decomposition, not a hypothetical one.
A minimal verifier-in-the-loop caught the failure using only the trace that already existed — no second model call, no re-doing the work. Checking one concrete fact (was the final output-producing tool actually called?) correctly passed the two successful conditions and failed the over-granular one, with a real, specific reason attached.

What plan-and-execute buys over a single loop¶

A ReAct-style agent interleaves reasoning and action in one continuous loop — every tool call happens after a fresh round of "what should I do next" reasoning from the same model. Plan-and-execute splits that: a planner produces the multi-step plan once, up front, and an executor works through it. LangChain's own framing gives three real advantages this buys over ReAct:

Speed — "they can execute multi-step workflow faster, since the larger agent doesn't need to be consulted after each action."
Cost — "they offer cost savings over ReAct agents. If LLM calls are used for sub-tasks, they typically can be made to smaller, domain-specific models."
Quality, from the act of planning itself — "they can perform better overall (in terms of task completions rate and quality) by forcing the planner to explicitly 'think through' all the steps required."

The mechanics of revision are explicit in the same framing: "Once execution is completed, the agent is called again with a re-planning prompt, letting it decide whether to finish with a response or whether to generate a follow-up plan (if the first plan didn't have the desired effect)." That's the theoretical case for replanning. Whether it's actually necessary — whether a model without an explicit replanning step still handles a broken plan assumption reasonably — is an empirical question, not a given.

Repro 1: a plan's assumption breaks mid-execution — does replanning matter?¶

The code below sets up the release-notes task and its two system prompts — one telling the model to execute its plan without second-guessing, one explicitly instructing it to revise on a broken assumption — against a repo that has never been tagged, so the plan's own premise is false before execution even starts.

_TAGS = {}  # "api-gateway" has NO tags -- this repo has never been tagged
_MERGED_PRS = {
    "api-gateway": [
        {"title": "Fix rate limiter off-by-one", "merged_at": "2026-08-02"},
        {"title": "Add request tracing headers", "merged_at": "2026-08-14"},
        {"title": "Upgrade TLS certificate rotation", "merged_at": "2026-09-01"},
    ],
}


def get_latest_tag(repo: str) -> dict:
    tag = _TAGS.get(repo)
    return {"tag": tag} if tag else {"tag": None, "note": "This repo has no tags yet."}


def list_merged_prs_since_tag(repo: str, since_tag: str) -> dict:
    if since_tag is None:
        return {"error": "since_tag is required and was not provided"}
    return {"prs": []}  # no tag exists, so "since tag X" can never match anything real


def list_all_merged_prs(repo: str) -> dict:
    return {"prs": _MERGED_PRS.get(repo, [])}


def draft_changelog(entries: list) -> dict:
    return {"changelog": "\n".join(f"- {e}" for e in entries)}

A real run — the same broken assumption, two different instructions, an identical real outcome

Task: "Prepare the release notes for the 'api-gateway' repo: find what's changed since the last release and draft a changelog." The natural plan: get the latest tag, list PRs merged since that tag, draft the changelog. The real trap: api-gateway has never been tagged — get_latest_tag returns {"tag": None, "note": "This repo has no tags yet."} — so the "since the last tag" half of the plan's premise is false before execution even starts.

Condition A ("execute it as planned rather than second-guessing each step"): real trace — get_latest_tag → list_all_merged_prs → draft_changelog. No attempt to force the now-broken list_merged_prs_since_tag call. Real answer: "This repository has no prior tags, so this changelog covers all merged PRs to date" — correct.

Condition B ("if a step's result reveals that an assumption behind your plan was wrong... stop and revise the rest of your plan"): identical real trace, identical correct outcome.

Both conditions produced the same three real tool calls and the same correct changelog, covering the same three real PRs. This is a genuine, honest negative result: whatever value an explicit replanning instruction adds in general, it didn't show up here, because Sonnet 5's baseline behavior already handled the broken assumption correctly without being told to.

Repro 2: the granularity tradeoff — a real, reproducible failure¶

The code below runs the identical release-notes task under three levels of plan granularity — no explicit structure, a well-sized four-step plan, and a plan forced into many small sub-steps — to see whether more upfront planning detail actually helps or costs something.

COARSE_SYSTEM = (
    "You are a release-notes assistant with access to tools. Complete the task in whatever way "
    "you judge best."
)

WELL_SIZED_SYSTEM = (
    "You are a release-notes assistant. Before using any tools, write out a plan with these "
    "steps: (1) find the latest tag, (2) if a tag exists, list PRs merged since it -- if no tag "
    "exists, list all merged PRs instead, (3) draft the changelog from what you found. Then "
    "execute that plan."
)

OVER_GRANULAR_SYSTEM = (
    "You are a release-notes assistant. Before using any tools, write out a highly detailed plan "
    "broken into as many small, specific sub-steps as possible -- for example, separate steps for "
    "connecting to the repository, authenticating, querying for the tag, parsing the tag response, "
    "checking whether the tag is null, querying for PRs, parsing each PR record individually, "
    "formatting each entry, and assembling the final draft. Write out every sub-step explicitly "
    "before executing anything, then execute your plan step by step."
)

A real run — the identical task, three decomposition levels, confirmed on two separate full runs

Too coarse (no explicit plan structure at all — "complete the task in whatever way you judge best"): real trace — 3 tool calls (get_latest_tag, list_all_merged_prs, draft_changelog), 4,951 tokens, a correct changelog.

Well-sized (four concrete, tool-mapped steps spelled out explicitly): real trace — the identical 3 tool calls, 5,441 tokens, a correct, slightly more detailed changelog.

Over-granular (a plan required to break the task into many small sub-steps — separate steps for "connecting to the repository, authenticating, querying for the tag, parsing the tag response, checking whether the tag is null..."): real trace — 0 tool calls. The real answer field is a cut-off plan document: "1. Repository connection & authentication... 2. Determine the last release point... 3. Retrieve merged PR data... 4. Parse PR results... 4.5. Compile a clean list" — the response hit its token budget mid-plan, before executing a single real action.

This isn't a one-off: the same three-way split — coarse succeeds, well-sized succeeds, over-granular produces zero tool calls — reproduced identically on a second, independent full run of the recipe. The mechanism is concrete, not philosophical: an exhaustively detailed upfront plan consumes the same token budget the executor needs to actually act, and a plan detailed enough can spend all of it before execution starts.

def verify_step(run: dict) -> dict:
    """A minimal verifier-in-the-loop check: did execution actually produce a real,
    checkable changelog, or just plan text with no tool calls behind it? This runs
    against the plan-and-execute loop's own real output, no extra model call needed --
    a verifier's job is to check the result that already happened, not to re-do it."""
    called_draft = any(t["tool"] == "draft_changelog" for t in run["trace"])
    if not called_draft:
        return {"passed": False, "reason": "draft_changelog was never called -- no real output was produced"}
    return {"passed": True, "reason": "draft_changelog was called; a real changelog was produced"}

The verifier, applied to all three real results: too_coarse → "passed: draft_changelog was called; a real changelog was produced." well_sized → passed, same reason. over_granular → "passed: false, reason: draft_changelog was never called — no real output was produced." One concrete check, against the trace each run already produced, no extra model call — and it correctly separated the two real successes from the one real failure.

What this means in practice¶

The two repros land in different places, and that's itself informative: replanning, in this run, turned out to be something Sonnet 5 already does without being told — the "broken assumption" scenario didn't need an explicit instruction to handle correctly. Granularity, by contrast, produced a real, repeatable failure with a concrete, checkable cause. The lesson isn't "replanning instructions are useless" or "always keep plans coarse" — it's that these two planning concerns have different real risk profiles for a current-generation model: one degrades gracefully by default, one has a real failure mode that specifically over-engineering the plan itself can trigger. A verifier-in-the-loop is cheap insurance for exactly the second case — it doesn't need to be a second, expensive model call second-guessing the first; checking one concrete fact about what actually happened (was the output-producing step reached at all?) is often enough to catch a real planning failure before it reaches whoever's waiting on the output.

Interview angle¶

Weak answer to "how would you make a multi-step agent more reliable?": "Have it write out a detailed plan before acting." This page's own real repro is a direct counterexample: the most detailed plan was the one that never got executed at all — the token budget spent articulating sub-steps was budget not spent calling the tools that would have actually produced the result.

Strong answer: plan detail has a real cost as well as a real benefit, and the right granularity is task-dependent, not "more is always safer." This page's own repro shows both directions of the correct principle in one comparison: too coarse didn't fail here (contrary to the intuition that an executor needs explicit guidance), but over-granular did fail, concretely, by consuming the execution budget on planning text. The actual discipline is sizing a plan step to be concrete enough to verify and large enough to be worth a distinct step — and checking that with a real run, the way this page's own recipe did, rather than assuming more detail is strictly better.

Follow-up to expect: "if a verifier just checks whether a tool was called, wouldn't a sufficiently clever over-granular plan game that check by calling a tool pointlessly?" Possibly — this page's own verifier is deliberately minimal (one concrete condition: was the actual output-producing step reached), which is exactly why it's cheap and worth having as a floor, not a ceiling. A production system would layer a real content check on top (does the produced changelog actually mention the right PRs, the way Evaluating Agents' outcome-vs-trajectory distinction argues for) — but even the minimal version already caught a real failure this page's own repro produced, for free, using only the trace that already existed.

Build it yourself — 30 minutes¶

Design a task with one tool whose result can plausibly invalidate the rest of a straightforward plan (this page's null-tag lookup is one pattern — a search that comes back empty, a record that doesn't exist, a status that's the opposite of expected). Run it once with a "commit to your plan" instruction and once with an explicit "revise on broken assumptions" instruction. Compare the real traces, not an assumed difference.
Take one real task and write three system prompts for it: no plan structure at all, a well-sized plan with as many steps as there are real tool calls, and a plan required to break every step into several smaller sub-steps. Run all three and compare real tool-call counts, real token totals, and whether each one actually finished.
Write one minimal verifier: a plain Python function checking one concrete, checkable fact about a completed run's trace (not re-running the task, not a second model call). Apply it to your own three granularity conditions and see whether it catches what actually went wrong.
If your own replanning test comes back a tie the way this page's own repro did, don't force a difference into the write-up — report it as what actually happened, and note what that implies about how much the explicit instruction was doing versus the model's own baseline behavior.

Scenario Check¶

Sources & further reading¶

LangChain, "Plan-and-Execute Agents" — the real, cited speed/cost/quality benefits over ReAct, and the replanning mechanism this page's repro 1 tests directly.
Reasoning Paradigms — ReWOO's dependency-via-variable-substitution mechanism, a related but distinct planning approach from plan-and-execute's serial tool calling.
Workflow Patterns — the evaluator-optimizer pattern, a related but distinct verification loop (content refinement) from this page's verifier-in-the-loop (execution-trace checking).
Evaluating Agents — outcome vs. trajectory grading, directly relevant to designing a more thorough verifier than this page's minimal one.