Awesome Testing
Request training offer
AI Learning Lab
0/8 complete

RAG lesson 01 · Define the retrieval contract

What must retrieval prove before generated prose can be trusted?

Versioned evidence corpus · deterministic retrieval fixtures

Lighthouse Policy Evidence Assistant. A fictional support assistant answers questions from a versioned product-policy corpus. It must retrieve current, authorized evidence, cite exact source spans, and abstain when evidence is missing or conflicting.

Prior knowledge
Basic LLM, vector, and API concepts help. Building with LLM APIs is useful context, but no vector-database experience is required.
Study time
3–4 hours guided · about 9 hours of optional deep dives
By the end
Design, evaluate, secure, and operate a RAG evidence pipeline from governed ingestion and hybrid retrieval through grounded answers, citations, and rollback.

01 · Smallest useful mechanism

Start with the claim, corpus, and relevance contract.

A RAG system joins ingestion, search, ranking, context assembly, generation, and evidence validation. Each stage can fail independently, so the product contract must name the answer claim and the evidence needed to support it.

Retrieval changes the model context; it does not make the resulting answer true.

02 · Experiment

Test the prediction

Deterministic retrieval fixture

This workbench uses inspectable ranking and evidence fixtures and injected failures. It does not make live calls or execute external effects.

Retrieval workbench · lighthouse-contract-v1

Choose the pipeline that can justify an answer about the current return policy.

Injected failure

A stale 14-day policy ranks above the current 30-day clause; a marketing FAQ promises 60 days.

Choose the implementation

The trace has not run.

Commit to an implementation, then inject the failure and inspect the actual state transitions.

03 · Implementation brief

Build it, break it, prove the outcome.

Implement
Write a retrieval contract naming corpus authority, supported questions, relevance, freshness, citation, and abstention rules.
Break it on purpose
Make a stale but lexically perfect policy outrank the current clause, then remove the current clause entirely.
Completion evidence
The first query selects the current authoritative span; the second abstains and records exactly which evidence requirement was unmet.

Your learning artifact

Write the argument you would defend

Stored only in this browser. No account required; course reset does not delete it.

04 · Check your understanding

What does a high semantic-similarity score establish?

Primary sources · verified 2026-08-02

What the fixture does not prove

A polished answer may cite stale, unauthorized, merely topical, or contradictory text. Without a retrieval contract, no metric or vector database configuration can say whether the product succeeded.

Common mistake: If the top chunk is semantically similar, the answer is grounded. Similarity estimates topical relation. Grounding requires current, authorized evidence that supports each material claim under the product’s rules.

Transfer exercise: Compare a topical marketing FAQ, a stale policy, and the current authoritative clause for the same question; decide what the product may claim.

Lewis et al. — Retrieval-Augmented GenerationThe original RAG paper formalizing generation with parametric memory plus retrieved non-parametric evidence and explicit provenance goals.Thakur et al. — BEIR benchmarkPrimary benchmark comparing lexical, sparse, dense, late-interaction, and reranking systems across heterogeneous retrieval tasks.OpenAI — Retrieval guideOfficial implementation contract for vector stores, semantic and hybrid search, attribute filters, rankers, thresholds, chunking, and result inspection.
Optional referenceDeep dive / reference chapterOpen the complete essay, diagrams, mathematics, exercises, glossary, and sources when you want more depth.

Next: A retrieval contract depends on a corpus whose parsing, chunking, identity, and deletion semantics preserve the source truth.