Stop letting your
AI guess.
Give it a library.

Two clean halves: understand the four pieces of RAG (embeddings → chunking → vector DB → retrieval) and then build a real "chat with your PDF" app. Every concept gets a simulator.

An LLM is a brilliant intern who finished training a year ago and was never allowed inside your office. RAG fixes both — fresh data, and your private data — by handing the intern a relevant page at question-time.

Forget the buzzwords. The whole pipeline is just a smart librarian sitting between your question and the model.

The 5-box pipeline (memorise this)

Boxes 1, 4, 5 you already know from Class 1. Boxes 2 + 3 are the entire job of today's class.

Before we can "search by meaning", we need a way to turn meaning into numbers. That's an embedding. And once you see what it does, every confusing thing about RAG suddenly makes sense.

Imagine a 2D world first — height & weight

Forget AI for a second. If I plot people by height vs weight, people of similar build end up close together on the chart. Same idea, scaled up. Real embeddings have 384, 768 or even 3072 dimensions — way more than we can draw — but the principle is identical: similar ⟶ close, different ⟶ far.

The aha: embedding math actually works

This is the demo that turned a generation of engineers into believers. Trained on enough text, vectors carry real relationships you can do arithmetic on. Pick a side and watch:

See it in 2D — same idea, drawable

A real-world plot of word vectors squished from 768 dimensions down to 2 for visualization. Click pairs and notice the direction connecting them is the same:

The score we use: cosine similarity

Two vectors close together = the angle between them is small. The cosine of that angle is our score. It ranges from -1 (opposite) to +1 (identical). Practically: above 0.7 means "very similar", below 0.3 means "barely related". No math needed to use it — just remember: bigger number = more similar.

Try it — the similarity meter

Type any two sentences. The meter shows the cosine similarity our system would compute (this in-browser simulation uses a small built-in semantic table, so results feel realistic for common topics).

How do we actually get an embedding? One function call.

You don't have to train anything — somebody else already did. Open-source models from Hugging Face (sentence-transformers) or APIs from OpenAI / Cohere give you embeddings in one line:

# pip install sentence-transformers
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("all-MiniLM-L6-v2")   # ① free, fast, 384 dims

vec = model.encode("A cat is sleeping on the couch")  # ② that's an embedding!
print(vec.shape)         # → (384,)  — 384 numbers
print(vec[:5])          # → [-0.05, 0.12, 0.41, -0.08, 0.22] (something like that)

# to compare two pieces of text → encode both → take cosine similarity
from numpy import dot
from numpy.linalg import norm

v1 = model.encode("A cat is sleeping on the couch")
v2 = model.encode("A kitten is napping on the sofa")
similarity = dot(v1, v2) / (norm(v1) * norm(v2))   # cosine
print(similarity)       # → 0.87 (very similar 🎉)

Before we embed anything, we have to cut it up. You can't embed a 200-page PDF as one vector — meaning gets averaged away to mush. So we split it into chunks. How you chunk decides how good your RAG actually is. Engineers underestimate this; the best ones obsess over it.

The 4 chunking strategies worth knowing

Watch all 4 strategies — same text, four different cuts

Same source passage, four ways to chop it. Switch tabs and watch where the cuts land and how chunks change shape. The differences are the lesson — pure text won't make this click; the picture will.

Try this sequence in class: start on Fixed at size 80 → count the 🪓 mid-word cuts → flip to Recursive at the same size → they're all gone → flip to Semantic → notice three chunks emerge that align perfectly with the three topics → flip to Structure → see how the markdown ## headers do the work for free.

One line of real code

from langchain_text_splitters import RecursiveCharacterTextSplitter

splitter = RecursiveCharacterTextSplitter(
    chunk_size=800,        # aim for ~800 chars per chunk
    chunk_overlap=100,     # adjacent chunks share 100 chars (context glue)
)

chunks = splitter.split_text(your_long_document)
print(len(chunks))    # → e.g. 47 chunks ready to embed

You have thousands of embeddings. For every question, you need the top-k nearest ones — fast. That's what a vector database does, and that's all it does.

The three names you'll hear all the time

What it actually does — three operations

We've talked about every piece. Now we wire them up. This is the whole pattern — every RAG system you'll see in production is just a fancier version of this.

Step 1 — Index your documents (once)

from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_huggingface import HuggingFaceEmbeddings
from langchain_chroma import Chroma

# ① load your documents (any text — for now, hardcoded)
docs = [
    "Our return policy allows refunds within 30 days of purchase.",
    "Shipping is free for orders above ₹999 across India.",
    "For corporate orders above 50 units, contact sales@example.com.",
    "Our office is in Indiranagar, Bangalore. Open Mon-Fri 10am-7pm.",
]

# ② split into chunks (small docs here, but production = thousands of pages)
splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=50)
chunks = splitter.create_documents(docs)

# ③ pick an embedding model (free, runs locally, no API key)
embedder = HuggingFaceEmbeddings(model_name="all-MiniLM-L6-v2")

# ④ build the vector store from chunks + embeddings (saves to disk)
db = Chroma.from_documents(chunks, embedder, persist_directory="./chroma_db")

print(f"Indexed {len(chunks)} chunks 🎉")

Step 2 — Ask a question (every time)

from langchain_huggingface import HuggingFaceEmbeddings
from langchain_chroma import Chroma
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from dotenv import load_dotenv
load_dotenv()

# ⑤ open the same vector store we built in step 1
embedder = HuggingFaceEmbeddings(model_name="all-MiniLM-L6-v2")
db       = Chroma(persist_directory="./chroma_db", embedding_function=embedder)
model    = ChatOpenAI(model="gpt-4o-mini", temperature=0)

# ⑥ the prompt: instructions + chunks + question
prompt = ChatPromptTemplate.from_template("""
Answer the question using ONLY the context below. If the context doesn't contain
the answer, say "I don't know." Be concise and quote facts directly.

Context:
{context}

Question: {question}
""")

def rag_answer(question):
    chunks = db.similarity_search(question, k=3)         # ⑦ retrieve top 3
    context = "\n\n".join(c.page_content for c in chunks)
    chain = prompt | model                                # ⑧ same chain trick as Class 2
    return chain.invoke({"context": context, "question": question}).content

print(rag_answer("How long do I have to return something?"))
# → "30 days from the purchase date."  ✅ from your data, not a guess

Watch it run — step-by-step pipeline

Press Next to see exactly what happens inside rag_answer("How long do I have to return something?"). Each step is a card; this is the whole RAG flow on one screen:

A vanilla RAG works. A good RAG works well. The two tricks that close that gap — and that every senior engineer asks about — are reranking and hybrid search. Both are easy to add.

Problem 1: the top-5 from vector search isn't the best 5

Embedding similarity is fast, but it sometimes ranks shallow word-matches above deep semantic matches. So we use a two-stage retrieval — a fast first pass to narrow down, then a slow accurate pass to pick the real winners.

Funnel total: ~210ms — fast enough for live chat, accurate enough to beat raw vector search by miles.

See reranking flip the order

Same query, same candidates. Left: what cosine similarity returned. Right: what a cross-encoder reranker returns. Press the button and watch the real answer rise:

The code to add reranking is laughably short:

from sentence_transformers import CrossEncoder

reranker = CrossEncoder("cross-encoder/ms-marco-MiniLM-L6-v2")   # free, fast

def retrieve_with_rerank(question, top_k=3):
    candidates = db.similarity_search(question, k=25)        # grab 25 cheap candidates
    pairs = [(question, c.page_content) for c in candidates]
    scores = reranker.predict(pairs)                          # cross-encoder scores each pair
    return [c for _, c in sorted(zip(scores, candidates), reverse=True)[:top_k]]

Problem 2: sometimes you need the exact word

Pure semantic search ignores keywords. Ask "what's the price of SKU-4429?" and the embedding doesn't care about that specific code — it just sees "price" and "product". For codes, names, IDs, dates — keyword search beats semantic hands down. The fix isn't to pick one or the other. It's to use both, with very different strengths.

Hybrid = both librarians on the case

Send the same query to both librarians at the same time. Each ranks the docs by their own logic. You then merge their two ranked lists into one using a tiny formula called Reciprocal Rank Fusion (RRF):

See it on a real query — "I want running shoes"

Same query, three rankings. Watch how Librarian A and Librarian B return different top picks — and how Hybrid merges them. Press the button to see who wins:

Time to ship. Upload any PDF — your resume, a research paper, an annual report, your college notes — and chat with it. Every answer cites the page it came from. This is the project that gets "how did you build this?" in your LinkedIn DMs.

The full app — 4 files

Step 1 — Load the PDF & index it once

# pip install langchain langchain-openai langchain-chroma langchain-huggingface \
#             langchain-community pypdf sentence-transformers gradio python-dotenv

from langchain_community.document_loaders import PyPDFLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_huggingface import HuggingFaceEmbeddings
from langchain_chroma import Chroma

def build_index(pdf_path):
    pages    = PyPDFLoader(pdf_path).load()                    # ① read all pages
    splitter = RecursiveCharacterTextSplitter(chunk_size=800, chunk_overlap=100)
    chunks   = splitter.split_documents(pages)                # ② chunk them
    embedder = HuggingFaceEmbeddings(model_name="all-MiniLM-L6-v2")
    db       = Chroma.from_documents(chunks, embedder)         # ③ in-memory store
    return db

Step 2 — Answer with retrieval + citations

from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from dotenv import load_dotenv
load_dotenv()

model = ChatOpenAI(model="gpt-4o-mini", temperature=0)

prompt = ChatPromptTemplate.from_template("""
You are a helpful PDF assistant. Answer the question using ONLY the context below.
If the context doesn't contain the answer, say "I couldn't find that in the document."
After your answer, list the page numbers you used as: Sources: page X, page Y.

Context:
{context}

Question: {question}
""")

def ask(db, question):
    chunks  = db.similarity_search(question, k=4)
    context = "\n\n".join(
        f"[page {c.metadata['page']+1]}] {c.page_content}" for c in chunks)
    chain = prompt | model
    return chain.invoke({"context": context, "question": question}).content

Step 3 — Give it a face with Gradio

import gradio as gr

state = {"db": None}                    # ① remember the index across turns

def upload(pdf):
    state["db"] = build_index(pdf.name)   # ② re-index whenever a new PDF arrives
    return "✅ PDF indexed! Ask me anything about it."

def chat(message, history):
    if state["db"] is None:
        return "Please upload a PDF first 📄"
    return ask(state["db"], message)

with gr.Blocks(title="📄 Chat with your PDF") as demo:
    gr.Markdown("## 📄 Chat with your PDF (powered by RAG)")
    pdf    = gr.File(label="Upload a PDF", file_types=[".pdf"])
    status = gr.Markdown()
    pdf.upload(upload, inputs=pdf, outputs=status)
    gr.ChatInterface(fn=chat)

demo.launch(share=True)                   # share=True → public link!

Run it

$ python pdf_chat.py

Running on local URL:  http://127.0.0.1:7860
Running on public URL: https://abcd12.gradio.live   # ← share this on LinkedIn!

Try the working version 👇

A live in-browser version with three pre-indexed sample documents — click a PDF, ask a question, see RAG retrieve the right chunk before answering:

⚙️ Simulated in-browser with a tiny semantic-matching layer so it runs key-free. Your real pdf_chat.py uses actual embeddings + GPT — same flow, same feel, larger brain.

"I built a RAG system" is an instant credibility upgrade — most people who post about AI have never actually built one. The shipping ritual continues: clip, caption, post, reply.

The 3-step ship checklist (same as Class 1 & 2)

Steal this caption

👉 When you type "@Shivank Agrawal", pick linkedin.com/in/shivankagrawal so the tag links correctly. Tagging your instructor and Scaler boosts reach & lets us amplify your post.

Want a variation? Pick a flavour, swap the PDF

The recipe is the same; the data makes it interesting. Try one of these on your own:

RAG works. But the frontier is making it smarter — and that's where this course leads next.

🤖 Agentic RAG — when retrieval becomes a decision

Today's RAG always retrieves. But sometimes you don't need to (small talk), and sometimes one retrieval isn't enough (multi-hop questions). Agentic RAG = combine Class 2's agent loop with Class 3's library. The agent decides: "do I need to look this up? Is what I found enough? Should I rewrite my query and try again?"

📈 Advanced RAG — the tricks the senior engineers use