Imagine you're considering a master's degree

You visit the UChicago ADS website and have a question: "How much does the program cost?" Instead of digging through dozens of web pages, you type it into a chat box — and get an instant, accurate answer with links to the source pages.

That's what this app does. It's a chatbot powered by AI that has 'read' every page on the ADS program website and can answer questions about admissions, courses, tuition, careers, and more.

What happens when you ask a question

From the outside, it looks simple: type → answer. But behind the scenes, your question goes on quite a journey.

This pattern has a name: RAG

This approach — searching for relevant information first, then letting AI compose an answer from it — is called RAG (Retrieval-Augmented Generation).

Check your understanding

Six actors, one mission

Think of this system like a hospital emergency room. Each specialist has a role — the triage nurse, the doctor, the lab — and they hand off patients (your question) to each other in a well-rehearsed sequence.

Where they live in the code

How the server boots up

When the server starts, it loads everything into memory so it's ready to answer questions instantly. This is the "opening the ER" moment — all equipment must be ready before the first patient arrives.

@asynccontextmanager
async def lifespan(app: FastAPI):
    embedder = OnnxEmbedder()
    cross_encoder = OnnxCrossEncoder()
    chunks = json.load(open("chunked_documents_dedup.json"))
    faiss_index = faiss.read_index("uchicago_ads_faiss_dedup.index")
    bm25 = BM25Okapi([tokenize_for_bm25(c["text"]) for c in chunks])
    llm = genai.Client(api_key=GOOGLE_API_KEY)

Check your understanding

The question's journey

Click "Next Step" to trace what happens when you ask "What are the admission requirements?"

If the components could text each other

Here's that same flow, but as a group chat. Watch how each component talks to the next one in the chain.

How the answer appears word by word

You've probably noticed the answer types out like someone is writing it in real time. This isn't a trick — the LLM generates one word at a time, and each word is sent to your browser immediately via a technique called SSE.

for chunk in response:
    token = chunk.text
    full_answer += token
    yield f"data: {json.dumps({'type':'token','content':token})}\n\n"

Check your understanding

From web pages to searchable knowledge

This process runs once, offline, before the app goes live. It transforms 147 raw web pages into 1,012 searchable text chunks.

Why you can't just feed whole pages to the AI

Chunking is one of the most important decisions in a RAG system. Here's why this project does it so carefully:

Three chunking strategies for three page types

Not all pages look the same. The system detects the page structure and picks the right strategy — like a filing clerk who handles books, folders, and index cards differently.

def extract_content(url, html):
    if _has_accordion(soup):
        # FAQ pages, course lists, schedules
        return extract_accordion_items(soup)
    elif _FAQ_URL_RE.search(url):
        # Q&A-style pages
        return extract_faq_items(soup)
    else:
        # Regular prose pages
        return split_by_paragraphs(text)

The surprising power of deduplication

Here's a plot twist: removing duplicate chunks was the single biggest improvement to search quality — bigger than the embedding model, bigger than reranking, bigger than everything else combined.

Check your understanding

Two ways to find what you're looking for

Imagine searching for a book in a library. You could search the catalog by keyword (title, author) or by topic (books about similar subjects). This system does both — and combines the results.

Combining results: Reciprocal Rank Fusion

Each search method returns its own ranking. RRF merges them into one master ranking where a chunk ranked high by both methods gets the best combined score.

for rank, idx in enumerate(faiss_order):
    rrf[idx] += 1.0 / (k + rank)
for rank, idx in enumerate(bm25_order):
    rrf[idx] += 1.0 / (k + rank)
# k=60 by default

Seven signals that fine-tune the ranking

After RRF fusion, the system applies extra signals — like a search engine giving bonus points to pages that match your intent perfectly.

The final judge: Cross-Encoder Reranking

The top 15 results go to a cross-encoder — a more powerful (but slower) model that reads the question and each chunk together to judge relevance. It keeps only the top 5.

def rerank(question, candidates, cross_encoder, top_k=5):
    pairs = [(question, c.text) for c in candidates]
    scores = cross_encoder.predict(pairs)
    ranked = sorted(zip(scores, candidates), reverse=True)
    return [c for _, c in ranked[:top_k]]

Check your understanding

Making AI models run fast on cheap hardware

The cross-encoder and embedder models originally ran on PyTorch — great for research, but slow on the affordable CPU servers this app runs on. The solution: convert them to ONNX format.

How ONNX inference works

class OnnxCrossEncoder:
    def __init__(self):
        self.session = ort.InferenceSession(
            "models/cross_encoder.onnx",
            providers=["CPUExecutionProvider"])
        self.tokenizer = AutoTokenizer.from_pretrained(
            "models/cross_encoder_tokenizer")

The deployment setup

The app runs on Google Cloud Platform with two services working together:

The cold start problem (and a clever fix)

Cloud Run can scale to zero — meaning if nobody asks a question for a while, the server shuts down. When the next user arrives, it takes 20-30 seconds to restart (load models, build index). This is called cold start.

The fix? Cloud Scheduler pings the server's health endpoint every 5 minutes, keeping it warm — like leaving the car engine running in a parking lot so it's ready when you need it.

# From deploy.sh
gcloud scheduler jobs create http keepalive \
  --schedule="*/5 * * * *" \
  --uri="$SERVICE_URL/health" \
  --http-method=GET

Check your understanding

The trap of "it looks right"

It's tempting to just ask a few questions, eyeball the answers, and call it done. But this project discovered something important: automated self-evaluation is dangerously optimistic.

The scorecard

These metrics come from 55 manually annotated test queries across 7 categories (admission, course, fee, capstone, career, application, contact) and RAGAS end-to-end evaluation.

The complete picture

You've now seen every piece of the puzzle. Here's how they all fit together:

What you can do now

After completing this course, you can:

Final challenge

You did it! 🎓

You now understand how a production RAG system works — from web scraping to streaming answers. Use this knowledge to build better AI applications and debug them with confidence.