Skip to content

What Is an Agent?

Hands-on

Full runnable recipe: what-is-an-agent/ in the companion cookbook — the same customer-support question run through a fixed pipeline and through a decision-point agent, with a real captured trace.

TL;DR — quick revision
  • The defining property isn't "uses an LLM." Anthropic's working definition: a workflow is a system where "LLMs and tools are orchestrated through predefined code paths"; an agent is a system where "LLMs dynamically direct their own processes and tool usage." The line is who decides the next step — fixed code, or the model.
  • A retry-on-failure loop written in application code is still a workflow, not an agent, under this definition — Anthropic's own evaluator-optimizer pattern (generate, grade, loop until a criterion is met) is listed as a workflow, because the LLM isn't the one deciding whether or how to retry.
  • Definitions genuinely disagree at the edges. Simon Willison: "an LLM agent runs tools in a loop to achieve a goal." Chip Huyen: something that perceives and acts on an environment. OpenAI: "systems that independently accomplish tasks on your behalf." None of these are wrong; they draw the line in slightly different places, and a strong interview answer names the disagreement rather than picking one as the definition.
  • Most systems marketed as agents aren't, by any of these definitions. Menlo Ventures' December 2025 enterprise survey found only 16% of enterprise and 27% of startup deployments calling themselves "agents" actually qualify as one — most are "if-then logic around a model call."
  • A worked trace beats a definition in an interview. This page's own demo shows the same misclassification happening in both a workflow and an agent — the difference isn't that the agent classifies better, it's that the agent has a step where it can act on its own doubt, and the workflow structurally doesn't.

The line, worked through an example

A plain LLM call is input in, output out — one shot, no loop, no branching on what comes back. The moment you add a loop, one question decides whether you've built a workflow or an agent: who decides what happens next?

If the answer is "the code I wrote" — a fixed sequence of steps, even one with retries or a quality check baked in — that's a workflow. Anthropic's Building Effective Agents names five workflow patterns (prompt chaining, routing, parallelization, orchestrator-workers, evaluator-optimizer) and is explicit that all five stay workflows even though several of them loop and even though some of them use an LLM to grade another LLM's output — because the decision to keep going is still governed by code the developer wrote (a criterion, a step count, a fixed router), not by the model choosing its own next action from live options.

If the answer is "the model, based on what it just observed" — that's an agent. The system's next action depends on the outcome of its own previous action, decided by the model itself at runtime, not read off a lookup table.

This is genuinely useful to apply, not just recite. Take a common interview scenario: a junior engineer builds a "customer support agent" that's really classify intent -> retrieve FAQ -> generate response. Every step can involve an LLM call. It is still a workflow — there's no point where the system can decide "this retrieval didn't actually answer the question, let me try something else." The moment you add that check, and let the model pick what happens next (broaden the search, ask a clarifying question, escalate), you've crossed into agent territory. That's exactly what this page's hands-on recipe builds, side by side, on the same task.

The two pipelines, in code

Same 5-entry FAQ knowledge base for both:

FAQ = {
    "refund_policy": "Refunds are available within 14 days of purchase for annual plans. Monthly plans are non-refundable.",
    "storage_limits": "The Free tier includes 5GB of storage. The Pro tier includes 2TB of storage.",
    "password_reset": "Reset your password from Settings > Security > Reset Password.",
    "data_export": "You can export all your files as a single ZIP from Settings > Export Data.",
    "account_deletion": "Deleting your account is permanent. Files are removed after a 30-day grace period.",
}

The workflow — classify -> retrieve -> generate, nothing can change what the next step does:

def workflow_answer(query: str, faq: dict, generate) -> dict:
    """The fixed pipeline: classify -> retrieve -> generate, no branching.
    Whatever category classify_intent picks, generate_response is stuck with --
    there's no step where the model can say "this isn't right, try something else."
    """
    categories = ", ".join(faq.keys())
    category_raw = generate(
        f"Categories: {categories}\n\nCustomer question: \"{query}\"\n\n"
        "Reply with exactly one category name from the list above. Reply with only the category name."
    ).strip().lower()
    category = next((c for c in faq if c in category_raw), category_raw)

    faq_entry = faq.get(category, "(no matching FAQ entry)")

    answer = generate(
        f"Customer question: \"{query}\"\n\nRelevant FAQ entry: \"{faq_entry}\"\n\n"
        "Write a short, direct reply to the customer using only the FAQ entry above."
    ).strip()

    return {"category": category, "faq_entry": faq_entry, "answer": answer}

The agent adds exactly two things the workflow doesn't have: a judge step, and a real decision point where the model — not an if/else in this code — picks what happens next.

def faq_answers_query(query: str, faq_entry: str, generate) -> bool:
    """The judge step the workflow never has: does the retrieved FAQ actually
    answer this question, or did retrieval just return the closest-sounding entry?
    """
    raw = generate(
        f"Customer question: \"{query}\"\n\nCandidate FAQ entry: \"{faq_entry}\"\n\n"
        "Does this FAQ entry actually answer the customer's question? Reply with only YES or NO."
    ).strip().upper()
    return raw.startswith("Y")

The judge step's sufficient: false verdict is what this next step actually reads before picking an action:

def decide_next_action(query: str, tried_categories: list, faq: dict, generate) -> str:
    """This is the decision point that makes it an agent rather than a workflow:
    the MODEL chooses the next step from live options, based on what just
    happened -- not a fixed `if/else` written into the pipeline code.
    """
    remaining = [c for c in faq if c not in tried_categories]
    raw = generate(
        f"Customer question: \"{query}\"\n\n"
        f"FAQ categories already checked and found NOT to answer this question: {tried_categories}\n"
        f"FAQ categories not yet checked: {remaining}\n\n"
        "Pick exactly one next action: BROADEN (try another FAQ category), "
        "CLARIFY (ask the customer a clarifying question), or ESCALATE (this "
        "needs a human). Reply with only one word: BROADEN, CLARIFY, or ESCALATE."
    ).strip().upper()
    for action in ("BROADEN", "CLARIFY", "ESCALATE"):
        if action in raw:
            return action
    return "ESCALATE"

That action choice — CLARIFY, in the run below — is what the loop then carries out:

def agent_answer(query: str, faq: dict, generate, max_steps: int = 3) -> dict:
    """The full agent loop: retrieve, judge, and if the judge says no, let the
    model itself decide what happens next -- broaden, clarify, or escalate.
    """
    tried_categories = []
    categories = ", ".join(faq.keys())
    category = generate(
        f"Categories: {categories}\n\nCustomer question: \"{query}\"\n\n"
        "Reply with exactly one category name from the list above. Reply with only the category name."
    ).strip().lower()
    category = next((c for c in faq if c in category), category)

    for step in range(1, max_steps + 1):
        faq_entry = faq.get(category, "(no matching FAQ entry)")
        sufficient = faq_answers_query(query, faq_entry, generate)

        if sufficient:
            answer = generate(
                f"Customer question: \"{query}\"\n\nRelevant FAQ entry: \"{faq_entry}\"\n\n"
                "Write a short, direct reply to the customer using only the FAQ entry above."
            ).strip()
            return {"outcome": "answered", "answer": answer, "steps": step}

        tried_categories.append(category)
        action = decide_next_action(query, tried_categories, faq, generate)

        if action == "ESCALATE":
            return {"outcome": "escalated", "answer": "Escalating to a human agent.", "steps": step}
        if action == "CLARIFY":
            return {"outcome": "clarify", "answer": "Could you say more about what you're trying to do?", "steps": step}

        remaining = [c for c in faq if c not in tried_categories]
        category = remaining[0] if remaining else category

    return {"outcome": "budget_exhausted", "answer": "(ran out of steps)", "steps": max_steps}

A real, unforced run — full input and output, both pipelines given the same misclassification

Input — the test query, identical for both pipelines: "My storage suddenly shows as full even though I deleted a bunch of files last week, and now I can't upload anything — is there some sync delay, or did I lose data?"

Both pipelines' classify_intent step picked storage_limits — a reasonable-sounding guess given the word "storage," and the wrong bucket for what's actually being asked (this isn't a tier-size question). That category resolves to this exact FAQ entry (from the FAQ dict shown above), which is what the generation step actually received as its only source material: "The Free tier includes 5GB of storage. The Pro tier includes 2TB of storage."

Output — workflow, stuck with that FAQ entry, replied in full:

"I don't have enough information in the provided FAQ entry to answer your question. The FAQ only states storage limits for different tiers, but doesn't address sync delays, data loss, or why your storage might still show as full after deletion.

To help you properly, I'd need access to FAQs or documentation covering: - How file deletion and storage updates work - Troubleshooting steps for storage issues - Whether there are known sync delays

I'd recommend contacting support directly so they can investigate your account specifically."

A genuinely honest finding worth stating plainly: Claude declined on its own here rather than confidently inventing an answer, and did so more explicitly than that — it named exactly what the FAQ doesn't cover and pointed the customer elsewhere. So the standard "naive pipeline confidently makes something up" framing doesn't automatically hold, and this held for both a small local model and a frontier one on this same query. What the workflow structurally still lacks, regardless of how well the wording comes out, is a mechanism to act on that doubt: whatever the generation step says, that's the final message back to the customer. There's no next step for it to take.

Output — agent, given the identical FAQ entry above, ran its judge step first. Input to that step was the query plus the same FAQ entry, and its output was a sufficient: false verdict — the model reading the same limited FAQ entry and, this time, saying so structurally instead of in prose. That verdict is what decide_next_action then reads, and its output was CLARIFY — the one real decision point in this recipe's design. (The clarifying question that gets sent back is a fixed reply in this minimal version, not model-generated text; what's real here is the action choice itself, not its wording — see the recipe's code above.)

Same input, same misclassification, same model. The difference isn't a smarter classifier — it's that the agent's architecture gave the model somewhere to go with its own uncertainty, and the workflow's didn't.

Definitions genuinely disagree — and that's fine to say out loud

There is no single canonical definition, and naming that is a stronger interview answer than confidently picking one:

  • Anthropic: workflows are "orchestrated through predefined code paths"; agents are where "LLMs dynamically direct their own processes and tool usage." Umbrella term: "agentic systems."
  • OpenAI: "Agents are systems that independently accomplish tasks on your behalf." Their Practical Guide to Building Agents frames it around three components: a model, tools, and instructions.
  • Simon Willison: "an LLM agent runs tools in a loop to achieve a goal." Deliberately mechanism-first, not autonomy-first.
  • Chip Huyen: an agent is anything that perceives its environment and acts on that environment through tools — closer to the classical AI definition, decoupling planning from execution.
  • LangChain: a spectrum rather than a binary, from a single tool call up to a fully autonomous loop.

Where they agree: an agent's next action has to depend on live feedback from its own previous action. Where they disagree: how much autonomy, and over how many steps, something needs before the label applies.

LLM agents vs. classical RL agents

Interviewers sometimes ask this to check you're not just pattern-matching "agent = has tools." A classical RL agent and an LLM agent share the loop structure (perceive, decide, act, observe, repeat) but differ in almost everything about how that loop gets good:

Classical RL agent LLM agent
Policy Learned via a reward signal over many episodes Mostly prompted, not trained, per task
Action space Fixed, often small and discrete Open-ended — any tool call, any text
Improvement mechanism Gradient updates from reward In-context learning, reflection, occasionally fine-tuning on trajectories (GRPO/DPO — see the Training section of this site)
Sample efficiency Needs many episodes Zero/few-shot, generalizes from pretraining
Typical failure mode Reward hacking, poor generalization outside training distribution Hallucinated actions, brittle to prompt drift, errors compounding over long horizons

RL-style training is increasingly applied on top of the LLM-agent loop rather than instead of prompting — that bridge is where an interviewer probing GRPO or DPO experience will usually head next.

A short history worth knowing

The "wave" of 2023 agent projects — AutoGPT, MetaGPT, ChatDev, AgentVerse, HuggingGPT — mostly wired an LLM into a fully autonomous loop with minimal guardrails and let it run. Several of those repositories are effectively stale today: interesting proofs of concept, not architectures anyone builds on directly now. What replaced that era wasn't a bigger loop; it was the workflow-vs-agent distinction itself (Anthropic's framing, December 2024) — the realization that most real tasks are better served by a bounded system that only reaches for full agentic autonomy where the task genuinely needs it, plus (from mid-2025 on) a second shift toward context engineering: treating what goes into the model's context on each step as the thing you actually design, not an afterthought once the loop is built. Both threads run through the rest of this site.

When you would not build an agent

This is a favorite trade-off question, and the strong answer leads with cost, not capability: agentic loops add latency (multiple model round-trips instead of one), cost (tokens scale with steps, and a stuck loop can burn a lot of them), non-determinism (harder to test, harder to guarantee a specific output shape), and a genuinely harder evaluation and debugging problem (see this site's Evaluating Agents page). If a task is well-scoped, doesn't need to branch on intermediate results, and a fixed pipeline already gets it right, a workflow is more reliable, cheaper, and easier to monitor — and reaching for an agent anyway is exactly the reflex interviewers are testing candidates for.

Use it for / skip it for

Reach for an agent when: the right sequence of steps genuinely can't be known in advance — the task is open-ended, multi-hop, or the correct next action depends on what a previous step actually returned (not just whether it succeeded).

Skip it when: the task is well-scoped and repetitive, the steps are known ahead of time, or a fixed pipeline with a quality check already handles the cases you see in practice — a workflow with an evaluator step is still simpler to reason about and monitor than an agent, even one that loops.

Interview angle

Weak answer: "An agent uses an LLM and a pipeline doesn't." Wrong — plenty of pipelines call an LLM at every stage and are still fixed pipelines.

Strong answer: name the decision-point distinction, give a concrete example of a pipeline that looks agentic but isn't (classify → retrieve → generate, no branching), and know at least two other framings exist (Willison's tool-loop, Huyen's perceive-and-act) without treating any one as universally "correct." Bonus: cite that most production systems marketed as agents don't actually meet this bar (Menlo's 16%/27% figure) — it signals you've looked past the marketing.

Follow-up to expect: "When would you not want to build something agentic, even though it's technically possible?" — see the section above. Interviewers like candidates who don't reflexively reach for an agent.

Build it yourself — 30 minutes

  1. Pick a small, narrow knowledge base (5 or so entries is enough) and a classify -> retrieve -> generate pipeline over it, using any provider — see the recipe for a working llm.py (Claude by default; a free local Ollama model also works).
  2. Find a query that breaks it — something with vocabulary mismatch, so classification picks the wrong bucket.
  3. Add exactly one thing: a judge step that asks the model "does this actually answer the question?" and, if not, a step where the model — not your code — picks the next action from a short menu (try again with a different category, ask a clarifying question, escalate).
  4. Run the same broken query through both versions and compare. You now have a working, honest answer to "show me the difference between a workflow and an agent" instead of a definition.

Scenario Check

Sources & further reading