What must retrieval prove before generated prose can be trusted?

Versioned evidence corpus · deterministic retrieval fixtures

Lighthouse Policy Evidence Assistant. A fictional support assistant answers questions from a versioned product-policy corpus. It must retrieve current, authorized evidence, cite exact source spans, and abstain when evidence is missing or conflicting.

Prior knowledge: Basic LLM, vector, and API concepts help. Building with LLM APIs is useful context, but no vector-database experience is required.
Study time: 3–4 hours guided · about 9 hours of optional deep dives
By the end: Design, evaluate, secure, and operate a RAG evidence pipeline from governed ingestion and hybrid retrieval through grounded answers, citations, and rollback.

Start here

What it is, when it helps, and where we are going.

Retrieval-augmented generation (RAG) is a pipeline that selects external evidence for a model at request time, then constrains or checks the generated answer against that evidence.

Good use cases

Answer questions from current, private, domain-specific, or frequently changing source collections.
Produce answers whose material claims can be linked to inspectable source spans and provenance.
Improve recall over large corpora while preserving abstention when authorized evidence is missing or conflicting.

What it does not solve

RAG is not automatically better than direct search, a database query, or a longer prompt. It adds value only when retrieval and generation jointly serve a defined evidence-backed answer contract.

Learning path

01Retrieval contract
02Corpus and chunks
03Candidate retrieval
04Context and generation
05Evaluation and operations

01 · Smallest useful mechanism

Start with the claim, corpus, and relevance contract.

A RAG system joins ingestion, search, ranking, context assembly, generation, and evidence validation. Each stage can fail independently, so the product contract must name the answer claim and the evidence needed to support it.

Retrieval changes the model context; it does not make the resulting answer true.

02 · Experiment

Test the prediction

Deterministic retrieval fixture

This workbench uses inspectable ranking and evidence fixtures and injected failures. It does not make live calls or execute external effects.

Retrieval workbench · lighthouse-contract-v1

Question to consider

Which pipeline can justify an answer about the current return policy?

Complication

A stale 14-day policy ranks above the current 30-day clause; a marketing FAQ promises 60 days.

What would you do?

Choose the approach you would be prepared to defend in a design review.

Send the top vector matchTake the highest similarity score and ask the model to answer confidently.Send all three documentsIncrease recall and let the model resolve the conflict.Filter, rank, and validate evidenceBind region and effective date, preserve spans, then answer or abstain.

Consider the trade-offs first.

Choose an answer, then run the scenario. You’ll see what the system checks, what happens under the complication, and where the control boundary holds or breaks.

03 · Implementation brief

Build it, break it, prove the outcome.

Implement: Write a retrieval contract naming corpus authority, supported questions, relevance, freshness, citation, and abstention rules.
Break it on purpose: Make a stale but lexically perfect policy outrank the current clause, then remove the current clause entirely.
Completion evidence: The first query selects the current authoritative span; the second abstains and records exactly which evidence requirement was unmet.

Your learning artifact

Write the argument you would defend

Stored only in this browser. No account required; course reset does not delete it.

Your response

04 · Check your understanding

What does a high semantic-similarity score establish?

Primary sources · verified 2026-08-02

What the fixture does not prove

A polished answer may cite stale, unauthorized, merely topical, or contradictory text. Without a retrieval contract, no metric or vector database configuration can say whether the product succeeded.

Common mistake: If the top chunk is semantically similar, the answer is grounded. Similarity estimates topical relation. Grounding requires current, authorized evidence that supports each material claim under the product’s rules.

Transfer exercise: Compare a topical marketing FAQ, a stale policy, and the current authoritative clause for the same question; decide what the product may claim.

Lewis et al. — Retrieval-Augmented GenerationThe original RAG paper formalizing generation with parametric memory plus retrieved non-parametric evidence and explicit provenance goals.Thakur et al. — BEIR benchmarkPrimary benchmark comparing lexical, sparse, dense, late-interaction, and reranking systems across heterogeneous retrieval tasks.OpenAI — Retrieval guideOfficial implementation contract for vector stores, semantic and hybrid search, attribute filters, rankers, thresholds, chunking, and result inspection.

Optional referenceDeep dive / reference chapterOpen the complete essay, diagrams, mathematics, exercises, glossary, and sources when you want more depth.on demand

Next: A retrieval contract depends on a corpus whose parsing, chunking, identity, and deletion semantics preserve the source truth.