Build Naive RAG From Scratch¶
A minimal, dependency-light naive RAG pipeline — PDF → chunk → embed → index → retrieve → generate — with no LangChain/LlamaIndex in the way, so every step stays visible. The code lives in rag-cookbook/naive-rag/; every block below is pulled live from those exact files at build time, so what you read here is always exactly what runs.
If you haven't yet, read Naive RAG first for the concepts — this page is the "now build it" companion, and deliberately skips re-explaining why each step exists.
What this deliberately keeps simple
Chunking here is plain fixed-size splitting, and the vector index is a brute-force FAISS flat index. Both are covered properly — with better alternatives — on the Chunking Strategies and Vector Databases pages. This tutorial is about the pipeline's shape, not about picking the best technique at each stage.
What you'll build¶
flowchart LR
A[PDF] --> B[Extract text]
B --> C[Chunk]
C --> D[Embed chunks]
D --> E[(FAISS index)]
F[Question] --> G[Embed question]
G --> E
E --> H[Top-k chunks]
H --> I[Build prompt]
I --> J[LLM]
J --> K[Answer]Sample data is "Attention Is All You Need" (Vaswani et al., 2017), downloaded at run time — a short, well-structured technical PDF that's freely available and widely used as sample data across RAG tutorials for exactly this reason. Swap in your own PDF any time (covered at the end).
1. Install¶
git clone https://github.com/DhruvMakwana/rag-cookbook.git
cd rag-cookbook/naive-rag
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
# Core pipeline
pypdf>=4.0
sentence-transformers>=3.0
faiss-cpu>=1.8
numpy>=1.26
python-dotenv>=1.0
requests>=2.31
# LLM providers — install whichever you plan to use (see .env.example).
# Ollama needs no SDK: the script talks to its local REST API over `requests`.
anthropic>=0.34
openai>=1.40
sentence-transformers pulls in PyTorch — expect a few hundred MB to download and a couple of minutes, one time only.
2. Configure your LLM¶
# Copy this file to .env in the same folder (or anywhere above it) and fill
# in only the keys for the provider(s) you plan to use. Never commit .env.
# "anthropic" | "openai" | "ollama"
LLM_PROVIDER=anthropic
ANTHROPIC_API_KEY=
ANTHROPIC_MODEL=claude-sonnet-4-5
OPENAI_API_KEY=
OPENAI_MODEL=gpt-4o-mini
# Ollama needs no API key — just a local server running.
OLLAMA_URL=http://localhost:11434/api/generate
OLLAMA_MODEL=llama3.1
# Optional: point at your own PDF instead of the default sample.
# PDF_PATH=/absolute/path/to/your.pdf
.env is git-ignored — it never gets committed, and the code never prints or logs its contents. Three interchangeable providers are supported, picked via LLM_PROVIDER:
- Anthropic or OpenAI — paste your key in, nothing else to configure.
- Ollama — fully local, no key needed, no cost. Install Ollama, run
ollama pull llama3.1once, setLLM_PROVIDER=ollama. Good for testing the pipeline before spending anything on a hosted API, or for readers following along without an API key at all.
3. Fetch the sample data¶
"""
Fetches a small sample PDF to run the naive RAG pipeline against.
Default: "Attention Is All You Need" (Vaswani et al., 2017) from arXiv —
a short (~15 page), well-structured technical PDF with headings, an
abstract, tables, and references. It's freely downloadable from arXiv
and is the same sample document used across most public RAG tutorials,
so it's a safe, reproducible default with no licensing ambiguity for
an educational demo like this one.
Swap in your own PDF any time — see the README ("Using your own PDF").
"""
import os
from pathlib import Path
import requests
DATA_DIR = Path(__file__).parent / "data"
DEFAULT_PDF_URL = "https://arxiv.org/pdf/1706.03762"
DEFAULT_PDF_PATH = DATA_DIR / "attention_is_all_you_need.pdf"
def download_sample_pdf(url: str = DEFAULT_PDF_URL, dest: Path = DEFAULT_PDF_PATH) -> Path:
"""Download the sample PDF if it isn't already cached locally."""
dest.parent.mkdir(parents=True, exist_ok=True)
if dest.exists() and dest.stat().st_size > 0:
print(f"Using cached PDF: {dest}")
return dest
print(f"Downloading sample PDF from {url} ...")
response = requests.get(url, timeout=30)
response.raise_for_status()
dest.write_bytes(response.content)
print(f"Saved to {dest} ({dest.stat().st_size / 1024:.0f} KB)")
return dest
if __name__ == "__main__":
pdf_url = os.environ.get("SAMPLE_PDF_URL", DEFAULT_PDF_URL)
download_sample_pdf(pdf_url)
Nothing exotic — download once, cache under data/ (git-ignored, so the PDF itself never ends up in the repo either). Point PDF_PATH at your own file instead any time; see Using your own PDF.
4. Extract text from the PDF¶
def load_pdf_text(pdf_path: Path) -> str:
"""Extract raw text from every page of a PDF and join it into one string."""
reader = PdfReader(str(pdf_path))
pages = [page.extract_text() or "" for page in reader.pages]
return "\n\n".join(pages)
pypdf walks every page and pulls out its text layer. This only works for PDFs with an actual text layer — a scanned document with no OCR would return empty strings here. (That failure mode is exactly what motivates Vision-RAG later on.)
5. Chunk it¶
def chunk_text(text: str, chunk_size: int = 800, overlap: int = 100) -> list[str]:
"""Fixed-size character chunking with overlap — the simplest possible
strategy, deliberately naive. See the Chunking Strategies page for
smarter options (recursive, semantic, small-to-big, ...)."""
chunks = []
start = 0
while start < len(text):
end = start + chunk_size
chunks.append(text[start:end])
start += chunk_size - overlap
return [c.strip() for c in chunks if c.strip()]
Fixed-size, character-based, with overlap — the simplest strategy that exists, on purpose. This is the one step you'll almost certainly want to upgrade first; see Chunking Strategies for what to reach for and when.
6. Embed the chunks¶
def embed_texts(texts: list[str], model: SentenceTransformer) -> np.ndarray:
"""Embed a list of strings and L2-normalize each vector, so a plain
inner-product search (used below) is equivalent to cosine similarity."""
embeddings = model.encode(texts, convert_to_numpy=True, show_progress_bar=False)
faiss.normalize_L2(embeddings)
return embeddings
all-MiniLM-L6-v2 via sentence-transformers — small (~80MB), fast on CPU, runs fully locally so indexing never needs an API key. Embeddings are L2-normalized so a plain inner-product search below is equivalent to cosine similarity, without computing norms at query time.
7. Build the vector index¶
def build_index(embeddings: np.ndarray) -> faiss.Index:
"""A flat (brute-force) FAISS index — exact search, fine at this scale
(a few hundred chunks). See Vector Databases for ANN indexes at scale."""
dimension = embeddings.shape[1]
index = faiss.IndexFlatIP(dimension)
index.add(embeddings)
return index
A flat FAISS index does brute-force exact search — perfectly fine at hundreds of chunks. It stops being fine in the millions; see Vector Databases for ANN indexes (HNSW, IVF) and when to reach for a real vector DB instead of an in-process index.
8. Retrieve¶
def retrieve(query: str, index: faiss.Index, chunks: list[str], embed_model: SentenceTransformer, k: int = 4) -> list[str]:
"""Embed the query the same way the chunks were embedded, then pull
back the k nearest chunks by cosine similarity."""
query_vec = embed_texts([query], embed_model)
_scores, indices = index.search(query_vec, k)
return [chunks[i] for i in indices[0]]
The query gets embedded with the same model used for the chunks (non-negotiable — see the embedding model page's one hard rule), then FAISS returns the k nearest chunks by cosine similarity.
9. Build the prompt¶
PROMPT_TEMPLATE = """Answer the question using ONLY the context below. \
If the context doesn't contain the answer, say you don't know — do not guess.
Context:
{context}
Question: {question}
Answer:"""
def build_prompt(question: str, retrieved_chunks: list[str]) -> str:
context = "\n\n---\n\n".join(retrieved_chunks)
return PROMPT_TEMPLATE.format(context=context, question=question)
Deliberately blunt: answer only from context, say "I don't know" rather than guess. No citation formatting, no chain-of-thought scaffolding — see Response Quality & Safety for hardening this properly.
10. Generate — pluggable across providers¶
def generate(prompt: str, provider: str | None = None, model: str | None = None) -> str:
"""Route to the configured LLM provider and return its text response."""
provider = (provider or os.environ.get("LLM_PROVIDER", "anthropic")).lower()
if provider == "anthropic":
return _generate_anthropic(prompt, model or os.environ.get("ANTHROPIC_MODEL", "claude-sonnet-4-5"))
if provider == "openai":
return _generate_openai(prompt, model or os.environ.get("OPENAI_MODEL", "gpt-4o-mini"))
if provider == "ollama":
return _generate_ollama(prompt, model or os.environ.get("OLLAMA_MODEL", "llama3.1"))
raise ValueError(f"Unknown LLM_PROVIDER '{provider}'. Use 'anthropic', 'openai', or 'ollama'.")
Each provider is one small function behind a common interface:
def _generate_anthropic(prompt: str, model: str) -> str:
import anthropic
api_key = os.environ.get("ANTHROPIC_API_KEY")
if not api_key:
raise RuntimeError("ANTHROPIC_API_KEY is not set. Add it to your .env file.")
client = anthropic.Anthropic(api_key=api_key)
response = client.messages.create(
model=model,
max_tokens=1024,
messages=[{"role": "user", "content": prompt}],
)
return response.content[0].text
def _generate_openai(prompt: str, model: str) -> str:
from openai import OpenAI
api_key = os.environ.get("OPENAI_API_KEY")
if not api_key:
raise RuntimeError("OPENAI_API_KEY is not set. Add it to your .env file.")
client = OpenAI(api_key=api_key)
response = client.chat.completions.create(
model=model,
messages=[{"role": "user", "content": prompt}],
)
return response.choices[0].message.content
def _generate_ollama(prompt: str, model: str) -> str:
import requests
url = os.environ.get("OLLAMA_URL", "http://localhost:11434/api/generate")
response = requests.post(
url,
json={"model": model, "prompt": prompt, "stream": False},
timeout=120,
)
response.raise_for_status()
return response.json()["response"]
Run it¶
First run downloads the PDF, chunks and embeds it, and caches a FAISS index under data/; later runs reuse that cache. Force a rebuild with --rebuild, change how many chunks get retrieved with --k, or override the provider per-call:
Example output (Anthropic, default settings):
$ python rag_naive.py "What is multi-head attention and why do they use 8 heads?"
Multi-head attention allows the model to jointly attend to information from
different representation subspaces at different positions. It works by
computing multiple attention heads in parallel (h=8 in this work), where
each head performs attention with its own learned projection matrices...
They use 8 heads with reduced per-head dimensions (dk = dv = dmodel/h = 64),
which keeps the total computational cost similar to single-head attention
with full dimensionality while gaining the representational benefit of
attending to multiple subspaces at once.
Using your own PDF¶
--rebuild is required the first time you point at a different PDF — otherwise the pipeline happily serves answers from whatever was cached under data/ last.
Where this breaks (and what fixes each break)¶
This is naive RAG — the failure modes are the point, not a bug list to feel bad about:
| Try asking... | What happens | Why | Fixed by |
|---|---|---|---|
| A vague or oddly-phrased question | Weak/irrelevant retrieval | Vocabulary mismatch between query and document phrasing | HyDE, query rewriting |
| A compound "compare X and Y" question | Retrieval blurs across both | One embedding can't represent two different information needs | Query decomposition |
| A question the PDF doesn't answer | A confident-sounding wrong answer, sometimes | Nothing checks whether retrieval actually found anything relevant | Self-RAG, CRAG |
| A question needing 2+ scattered facts | Right facts retrieved, ranked poorly, one gets cut off at k |
No re-ranking, no compression, fixed top-k | Reranking, compression |
Full failure-mode discussion: Naive RAG.