Skip to content

PageIndex

Every technique on this site so far relies on either embeddings and vector similarity search, or precomputed context caching. PageIndex rejects the vector-similarity premise entirely, for a specific class of documents: instead of chunking and embedding, it turns a document's actual structure — headers, sections, table of contents — into a tree, and has an LLM navigate that tree to find the right section. The idea is closer to how a person finds an answer in a long report: flip to the right chapter based on the document's own organization, not scan every page for text that looks similar to the question.

Scope: what this page actually measures

Vector RAG's edge on plain, shallow documents (this site's usual "Attention Is All You Need" sample) is real — that paper has ~7 top-level sections and barely any sub-numbering, which gives a tree-navigation approach almost nothing to navigate. This page uses a different sample document on purpose: NIST's "Artificial Intelligence Risk Management Framework (AI RMF 1.0)", a real, free, 48-page regulatory document with genuine 4-level structure (Part → Section → Subsection → numbered category). It's the kind of document this technique is actually built for, and using it here is what makes the comparison below meaningful rather than favorable.

Hands-on

The full comparison below is runnable, with real API calls: PageIndex → in the code repo, using the real open-source pageindex package. Needs an Anthropic API key; retrieval and generation both run through it.

TL;DR — quick revision
  • No embeddings, no vector database: a document's real structure becomes a tree; an LLM reasons over section titles and summaries to decide where to look, then pulls the actual text from just those sections
  • A different sample document than every other page on this site, deliberately: the usual paper's shallow structure wouldn't exercise this technique's actual advantage, so this page uses a genuinely deep, 4-level-structured regulatory PDF instead
  • Measured directly: PageIndex 5/5 correct with accurate page citations, naive text RAG 3/5, on a 5-question eval set built specifically for this document
  • The two naive-RAG misses have a specific, checked cause: embedding similarity retrieved generically similar-sounding prose elsewhere in the document instead of the exact numbered subsection or exact appendix text — precisely the failure mode this technique targets
  • The real cost: each PageIndex query here took 6-12 seconds (an actual reasoning loop over the tree), against naive RAG's well under a tenth of a second (one cosine-similarity lookup) — a real latency trade-off for the accuracy gain, not a free win

How it works

Tree construction. A layout parser extracts the document's actual structure — headings, table of contents — directly, without an LLM, where possible. An LLM is used for two smaller things: confirming a detected heading really starts where the parser thinks it does, and writing each section's summary. For the 48-page sample document here, indexing took about a minute and cost a few cents.

Retrieval and generation, together. Given a question, the chat model is handed the tree — section titles and summaries, not full text — reasons about which section(s) look relevant, pulls the actual text for just those sections, and answers, looping back for more if the first ones weren't enough. This is an agentic tool-calling loop under the hood, not a single classification call: the model can go deeper into a promising branch, back out of an unpromising one, and only reads full section text once it has a specific place to look.

def index_document(pdf_path: str) -> str:
    """Build the document's tree. Structure detection (headers, sections,
    table of contents) happens without an LLM where possible -- a layout
    parser reads it directly from the PDF. An LLM is used for two
    smaller things: verifying that a detected heading really starts on
    the page the layout parser thinks it does, and writing each node's
    summary. Returns a doc_id used for every later query against this
    document -- indexing happens once, not per query."""
    from pageindex import PageIndexClient

    client = PageIndexClient(
        index_model="anthropic/claude-haiku-4-5",
        chat_model="anthropic/claude-sonnet-5",
        storage_path="./.pageindex",
    )
    result = client.submit_document(pdf_path)
    return result["doc_id"]

def pageindex_search(question: str, doc_id: str) -> tuple:
    """Retrieval and generation happen together in one call: the LLM is
    given the document's tree (section titles and summaries, not full
    text), reasons about which section(s) likely contain the answer,
    pulls the actual text for just those sections, and answers --
    looping back for more sections if the first ones weren't enough.
    `citations=True` returns which pages it actually used, so you can
    verify it navigated to the right place rather than trusting the
    answer blindly."""
    import re

    from pageindex import PageIndexClient

    client = PageIndexClient(
        index_model="anthropic/claude-haiku-4-5",
        chat_model="anthropic/claude-sonnet-5",
        storage_path="./.pageindex",
    )
    answer = client.chat(question, doc_id=doc_id, citations=True)
    pages = sorted({int(m) for m in re.findall(r'<cite doc="[^"]*" page="(\d+)"', answer)})
    clean_answer = re.sub(r'\s*<cite doc="[^"]*" page="\d+"/>', "", answer)
    return clean_answer, pages

Compared directly against the naive text RAG baseline used elsewhere on this site — extract text, chunk it, embed with a dedicated text embedder, retrieve by cosine similarity:

def naive_text_search(pdf_path: str, query: str, k: int = 3) -> list:
    """The baseline this page compares against: extract text, chunk it,
    embed chunks with a dedicated text embedder, retrieve by cosine
    similarity. No document structure is used at all -- a chunk from the
    middle of one section looks the same to this pipeline as a chunk
    from the start of another."""
    from langchain_text_splitters import RecursiveCharacterTextSplitter
    from pypdf import PdfReader
    from sentence_transformers import SentenceTransformer, util

    reader = PdfReader(pdf_path)
    full_text = "\n\n".join(page.extract_text() or "" for page in reader.pages)
    splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=50)
    chunks = splitter.split_text(full_text)

    model = SentenceTransformer("all-MiniLM-L6-v2")
    chunk_embeddings = model.encode(chunks, convert_to_numpy=True, show_progress_bar=False)
    query_embedding = model.encode(query, convert_to_numpy=True)
    sims = util.cos_sim(query_embedding, chunk_embeddings)[0]
    top_k = sims.topk(min(k, len(chunks)))
    return [(chunks[i], sims[i].item()) for i in top_k.indices.tolist()]

Measured: PageIndex 5/5, naive text RAG 3/5

A 5-question eval set specific to this document — each keyword checked directly against the real page text, not guessed:

                                                              Naive   PageIndex (pages cited)
What is Risk Tolerance adapted from?                          PASS    PASS  [12]
How is GOVERN described relative to other functions?          PASS    PASS  [25,26,27]
What does MEASURE 2.7 evaluate?                                FAIL    PASS  [35]
Per Appendix B, are AI risks comprehensively addressed?        FAIL    PASS  [43,44]
Is it easy to tell if an opaque AI system is fair/secure?      PASS    PASS  [21]

Every PageIndex page citation was checked against the document's real tree structure, and each one is accurate — it navigated to the actual section containing the answer, not just a plausible-sounding one.

Why the two naive-RAG misses happened, checked directly

For "What does MEASURE 2.7 evaluate?", the top-3 retrieved chunks were a nearby MAP-function table and generic test/evaluation/validation process text — semantically close to the query's own vocabulary ("measure", "evaluate"), but not the actual MEASURE 2.7 entry. The same pattern held for the Appendix B question: the retrieved chunks were generic risk-management prose from elsewhere in the document, not Appendix B's specific language. In both cases, embedding similarity pulled toward text that sounds like the query rather than the document's own structural answer. PageIndex doesn't have this failure mode by construction — it isn't comparing the query's wording against candidate text at all, it's reasoning about which section of the document's own outline should contain the answer.

The real cost: latency, not measured here in dollars

Retrieval and generation happen as one agentic call in PageIndex, and the library doesn't expose a clean per-call token/cost breakdown the way a raw API call does (see the CAG page for that kind of accounting on a different technique). What is directly measured: each PageIndex query here took 6-12 seconds, against naive RAG's well under a tenth of a second — a real difference, since PageIndex is running an actual multi-step reasoning loop over the tree rather than one embedding lookup. That latency (and the real, if unmeasured, extra token cost of the agentic loop) is the genuine price of the accuracy gain above, not a side effect to ignore.

Scaling beyond one document

PageIndex's method targets long, individually-structured documents — but a knowledge base of many such documents (a library of contracts, a shelf of regulatory filings) doesn't automatically extend to "build one giant tree over everything." Each document keeps its own tree; picking which document to search within first is a separate, coarser routing problem — the same idea as choosing which retrieval path to take, covered on the Modular RAG page, or which cached document to load, covered on the CAG page. This page's own comparison is deliberately about retrieval within one document, where PageIndex's structural-navigation advantage is most directly testable.

When to use which

Naive vector RAG PageIndex
Best fit Loosely structured or short documents — chat logs, tickets, memos with no real hierarchy Long, well-structured professional documents — contracts, filings, regulatory/technical manuals with genuine section structure
What it needs An embedding model, a vector index Just the document's own structure (headers, TOC) plus an LLM
Per-query cost One embedding + one similarity search — fast and cheap An LLM reasoning loop over the tree — measured here at 6-12s per query, real token cost beyond that
Failure mode Retrieves text that sounds like the query, missing the document's own exact answer (measured directly above) Needs genuine exploitable structure — a structureless document gives it nothing to navigate
Explainability An opaque similarity score A page-citation trail showing exactly which sections were used, checkable against the document's real structure

Use PageIndex for: long, genuinely structured professional documents, especially where "why did the system think the answer was here" needs a real answer for compliance or trust reasons, not just a similarity score. Use vector RAG for: large corpora of short or loosely-organized documents, or anywhere per-query latency and cost need to stay minimal. Neither cleanly wins at synthesis questions spanning many unrelated sections at once — that's a closer fit for the relational, multi-source problem Graph RAG is built for.

Scenario Check

Sources & further reading

  • VectifyAI/PageIndex — the real, open-source library used in this recipe, local mode
  • PageIndex documentation — the tree-search reasoning pattern and API reference
  • Mafin2.5 / FinanceBench results — PageIndex's own published benchmark on financial-filing QA, for a second data point beyond this page's own measurement
  • CAG — another technique measured on real API cost/latency numbers, contrasted with this page's own (partial) cost accounting
  • Modular RAG and Graph RAG — routing across multiple documents and multi-source synthesis, both referenced above as the problems PageIndex's single-document tree doesn't solve on its own