Skip to content

The Agent Loop From Scratch

Hands-on

Full runnable recipe: agent-loop-from-scratch/ in the companion cookbook — a real tool-calling loop with the raw Anthropic client, no framework, checked against an independently computed ground truth.

TL;DR — quick revision
  • The model never calls your function. At inference time it's given a schema and generates a structured request — a tool_use block naming a tool and its arguments. Your orchestration code parses that, runs the real function, and sends the result back as a new message (a tool_result) on the next turn. All the "agentic" behavior here is: structured generation, external execution, and a feedback loop — nothing more magical than that.
  • The model has no guaranteed connection to reality until your code creates one. It can request a tool that doesn't exist, or arguments that don't validate — there's no built-in check that its output matches the schema until your orchestration code checks.
  • Tool descriptions are a prompt, not documentation. Vague parameter descriptions produce vague argument extraction; concrete descriptions with worked examples measurably improve it. This is a design surface, not an afterthought.
  • A hard iteration cap bounds round-trips, not per-call latency. They're separate risks and need separate limits — a loop that can't run away forever can still stall for a long time on one slow tool call.
  • A real run here produced a genuinely counter-intuitive result: the no-tools baseline got every arithmetic step right on its own — what it actually lacked was a live exchange rate, which isn't an arithmetic problem at all.

Mechanism: what "the model calls a tool" actually means

The weak version of this explanation — the one that gets corrected in an interview — is "the model calls the function." It doesn't, and there's no version of any major provider's API where it does.

What actually happens: your code sends the model a list of tool schemas (name, description, a JSON Schema for the arguments) alongside the conversation. If the model decides a tool would help, it doesn't execute anything — it generates a structured block saying, in effect, "I'd like to call calculate with expression: '127.50 * 1.18'." That's still just text generation, constrained to a particular shape. Your code reads that block, actually runs calculate("127.50 * 1.18") for real, and sends the return value back to the model as a new message. The model reads that result on its next turn and decides what to do next — answer, or request another tool. Anthropic's response shape makes this explicit: stop_reason: "tool_use" marks a turn as a request rather than an answer, and the result has to travel back as its own tool_result message, not get silently appended anywhere.

This is the entire mechanism. There's no hidden channel, no direct invocation, and critically: no guarantee. The model's tool request is just generated text that happens to be shaped like a function call — it can name a tool that doesn't exist, or supply arguments that don't match the schema, and nothing stops it from doing so except your code checking.

TOOLS = [
    {
        "name": "calculate",
        "description": (
            "Evaluate a numeric arithmetic expression using +, -, *, /, ** and parentheses. "
            "Example: '127.50 * 1.18' to add an 18% tip to 127.50. "
            "Numbers only -- no currency symbols, no words."
        ),
        "input_schema": {
            "type": "object",
            "properties": {"expression": {"type": "string", "description": "e.g. '150.45 / 4'"}},
            "required": ["expression"],
        },
    },
    {
        "name": "convert_currency",
        "description": (
            "Convert an amount from one currency to another using this app's fixed exchange "
            "rate table (illustrative rates, not live data). "
            "Example: convert_currency(150.45, 'EUR', 'USD')."
        ),
        "input_schema": {
            "type": "object",
            "properties": {
                "amount": {"type": "number"},
                "from_currency": {"type": "string", "description": "3-letter code, e.g. 'EUR'"},
                "to_currency": {"type": "string", "description": "3-letter code, e.g. 'USD'"},
            },
            "required": ["amount", "from_currency", "to_currency"],
        },
    },
]

def run_agent_loop(question: str, client, model: str, tools: list, tool_impl: dict, max_iterations: int = 6) -> dict:
    """The real loop. Each iteration: send the conversation so far, and if the
    model asked for a tool (stop_reason == "tool_use"), actually run it and
    send the result back as a new message -- the model never touches the
    tool directly, this loop is the only thing that does."""
    import json
    messages = [{"role": "user", "content": question}]
    trace = []

    for step in range(1, max_iterations + 1):
        response = client.messages.create(model=model, max_tokens=1024, tools=tools, messages=messages)
        messages.append({"role": "assistant", "content": response.content})

        if response.stop_reason != "tool_use":
            final_text = "".join(b.text for b in response.content if b.type == "text")
            return {"outcome": "answered", "answer": final_text, "trace": trace, "steps": step}

        tool_results = []
        for block in response.content:
            if block.type != "tool_use":
                continue
            fn = tool_impl[block.name]
            result = fn(**block.input)
            trace.append({"step": step, "tool": block.name, "arguments": block.input, "result": result})
            tool_results.append({"type": "tool_result", "tool_use_id": block.id, "content": json.dumps(result)})
        messages.append({"role": "user", "content": tool_results})

    # Hard budget, hit: return honestly rather than looping forever.
    return {"outcome": "budget_exhausted", "answer": "(ran out of steps)", "trace": trace, "steps": max_iterations}

Design principles: the tool schema is a prompt

Two real tools back this page's demo. The first is a genuinely safe calculator — not eval() on whatever text the model produces, but an AST walk that only permits numeric literals and + - * / **, so a malformed or adversarial expression can't do anything beyond arithmetic:

import ast
import operator

_SAFE_OPS = {
    ast.Add: operator.add, ast.Sub: operator.sub,
    ast.Mult: operator.mul, ast.Div: operator.truediv,
    ast.Pow: operator.pow, ast.USub: operator.neg,
}


def _safe_eval(node):
    """Walk a parsed expression, allowing only numeric literals and +-*/**."""
    if isinstance(node, ast.Expression):
        return _safe_eval(node.body)
    if isinstance(node, ast.Constant) and isinstance(node.value, (int, float)):
        return node.value
    if isinstance(node, ast.BinOp) and type(node.op) in _SAFE_OPS:
        return _SAFE_OPS[type(node.op)](_safe_eval(node.left), _safe_eval(node.right))
    if isinstance(node, ast.UnaryOp) and type(node.op) in _SAFE_OPS:
        return _SAFE_OPS[type(node.op)](_safe_eval(node.operand))
    raise ValueError(f"Disallowed expression: {ast.dump(node)}")


def calculate(expression: str) -> dict:
    """A real calculator. Not eval() on model output -- an AST walk that only
    allows numbers and +, -, *, /, ** so a malformed or malicious expression
    can't do anything beyond arithmetic."""
    try:
        tree = ast.parse(expression, mode="eval")
        return {"result": _safe_eval(tree)}
    except Exception as e:
        return {"error": f"Could not evaluate '{expression}': {e}"}

The second is a currency converter over a fixed, illustrative rate table:

# Fixed, illustrative exchange rates -- not live data, and the tool
# description says so, so the model doesn't treat this as authoritative.
_RATES = {
    ("EUR", "USD"): 1.08, ("USD", "EUR"): 1 / 1.08,
    ("GBP", "USD"): 1.27, ("USD", "GBP"): 1 / 1.27,
    ("USD", "USD"): 1.0,
}


def convert_currency(amount: float, from_currency: str, to_currency: str) -> dict:
    key = (from_currency.upper(), to_currency.upper())
    if key not in _RATES:
        return {"error": f"No exchange rate available for {from_currency} -> {to_currency}"}
    return {"result": round(amount * _RATES[key], 4)}

Notice both tool descriptions carry a worked example ('127.50 * 1.18' to add an 18% tip, convert_currency(150.45, 'EUR', 'USD')), not just a type signature. That's deliberate: a schema that only says expression: string leaves the model guessing at format; a schema with a concrete example in the description measurably improves how reliably the model extracts the right arguments. The description is doing real work — it's the closest thing this system has to a spec the model actually reads.

The other design choice worth naming: calculate's except branch doesn't crash the loop or swallow the problem silently — it returns a structured {"error": "..."} dict, which becomes a normal tool_result the model can read and act on. A raw stack trace or a silent failure both leave the model unable to tell what went wrong; a structured error message lets it decide whether to retry with a fixed expression, try a different tool, or give up and say so.

Stop conditions and budgets

The loop in this recipe is capped at a fixed number of iterations, and if the model still hasn't finished, it returns an honest "ran out of steps" result instead of looping forever:

def run_agent_loop(question: str, client, model: str, tools: list, tool_impl: dict, max_iterations: int = 6) -> dict:
    """The real loop. Each iteration: send the conversation so far, and if the
    model asked for a tool (stop_reason == "tool_use"), actually run it and
    send the result back as a new message -- the model never touches the
    tool directly, this loop is the only thing that does."""
    import json
    messages = [{"role": "user", "content": question}]
    trace = []

    for step in range(1, max_iterations + 1):
        response = client.messages.create(model=model, max_tokens=1024, tools=tools, messages=messages)
        messages.append({"role": "assistant", "content": response.content})

        if response.stop_reason != "tool_use":
            final_text = "".join(b.text for b in response.content if b.type == "text")
            return {"outcome": "answered", "answer": final_text, "trace": trace, "steps": step}

        tool_results = []
        for block in response.content:
            if block.type != "tool_use":
                continue
            fn = tool_impl[block.name]
            result = fn(**block.input)
            trace.append({"step": step, "tool": block.name, "arguments": block.input, "result": result})
            tool_results.append({"type": "tool_result", "tool_use_id": block.id, "content": json.dumps(result)})
        messages.append({"role": "user", "content": tool_results})

    # Hard budget, hit: return honestly rather than looping forever.
    return {"outcome": "budget_exhausted", "answer": "(ran out of steps)", "trace": trace, "steps": max_iterations}

Worth being precise about what this cap does and doesn't cover: it bounds how many round-trips the loop can make, which is what keeps a stuck agent from silently burning an unbounded number of API calls. It says nothing about how long any single tool call is allowed to run — a tool that hangs for 90 seconds isn't touched by an iteration cap at all. Production loops need both: a step budget on the loop, and an independent timeout on each tool execution.

A real run — one honest surprise, full input and output

Input, sent identically to both versions: "I'm splitting a €127.50 dinner bill among 4 friends, and we want to add an 18% tip on top before splitting. What does each person owe, in US dollars?" Ground truth, computed independently in plain Python, not by the model: $40.62.

Output, no tools — in full, not trimmed:

Step 1: Calculate the tip — EUR127.50 × 0.18 = EUR22.95

Step 2: Total with tip — EUR127.50 + EUR22.95 = EUR150.45

Step 3: Split among 4 people — EUR150.45 ÷ 4 = EUR37.61 per person

Step 4: Convert to USD — EUR37.61 × current exchange rate

Each person owes approximately $40.80–$41.50 USD (depending on the current EUR/USD exchange rate, which fluctuates daily)

Note: I've used a typical exchange rate around 1.08–1.10 USD per EUR. Check a current converter for the exact amount with today's rate.

Every arithmetic step there is correct. The only thing it got wrong was guessing at an exchange rate instead of having a real one.

Output, with tools — the exact JSON this recipe's loop returns:

{
  "outcome": "answered",
  "answer": "Each person owes approximately $40.62 USD.\n\nHere's the breakdown:\n- Original bill: EUR 127.50\n- With 18% tip: EUR 150.45\n- Per person (÷4): EUR 37.61\n- Converted to USD: **$40.62 per person**",
  "trace": [
    {"step": 1, "tool": "calculate", "arguments": {"expression": "127.50 * 1.18"}, "result": {"result": 150.45}},
    {"step": 1, "tool": "calculate", "arguments": {"expression": "127.50 * 1.18 / 4"}, "result": {"result": 37.6125}},
    {"step": 2, "tool": "convert_currency", "arguments": {"amount": 37.6125, "from_currency": "EUR", "to_currency": "USD"}, "result": {"result": 40.6215}}
  ],
  "steps": 3
}

Reading the trace: both calculate calls landed in step: 1 — the model issued them in parallel, in the same turn, without being asked to. The second call is mildly redundant (it recomputes 127.50 * 1.18 again as part of 127.50 * 1.18 / 4, instead of reusing the first call's 150.45) — correct, but not the shortest path available. step: 2 then converts the correct EUR-per-person figure to USD, landing on the ground truth exactly.

The genuinely useful finding here isn't "the baseline can't do math" — the trace above shows it could. It's that the gap was a live-data problem, not an arithmetic one, and no amount of arithmetic skill closes a live-data gap without a real source for that data.

Interview angle

Weak answer to "walk me through exactly what happens between the model deciding to call a tool and the tool's result reaching the model again": "the model calls the API." This skips the entire mechanism and reads as not having thought about it.

Strong answer: name the two-message round trip explicitly — the model generates a structured tool_use request (still just text generation, constrained to a schema), your orchestration code parses and executes it for real, and the result comes back as a distinct tool_result message the model reads on its next turn. Mention that nothing guarantees the request is valid until your code checks.

Follow-up to expect: a version of "your agent has 40 tools registered and keeps picking the wrong one or inventing arguments — what do you do?" A weak instinct is to collapse many tools into one mega-tool with a free-text instruction field. The stronger answer notices that this doesn't remove the selection problem, it just moves it from the model (choosing among well-described, schema-validated tools) into your own code (a keyword matcher choosing among internal actions from unstructured text) — trading a visible, debuggable failure mode for a hidden one, and losing schema validation in the process.

Build it yourself — 30 minutes

  1. Pick two small, deterministic tools whose outputs you can check independently (a calculator and a unit or currency converter both work well — you want a task where "did the model get this right" isn't a matter of taste).
  2. Write the raw loop: send the conversation, check stop_reason, execute any requested tool for real, append the result as a new message, repeat with a hard iteration cap.
  3. Ask the same question with no tools at all first, and write down what the model says.
  4. Run it again with tools, and compare against an answer you computed independently, by hand or in a separate script — not by asking the model to check its own work.
  5. Look at why the two answers differ, not just whether they differ. It's rarely as simple as "no tools means wrong."

Scenario Check

Sources & further reading