How should generation turn evidence into an answer without inventing authority?

Versioned evidence corpus · deterministic retrieval fixtures

Lighthouse Policy Evidence Assistant. A fictional support assistant answers questions from a versioned product-policy corpus. It must retrieve current, authorized evidence, cite exact source spans, and abstain when evidence is missing or conflicting.

01 · Smallest useful mechanism

Make claim support and abstention observable product states.

Grounded generation constrains the answer to selected evidence, attaches stable citations, and validates material claims. It must handle insufficient or conflicting evidence without asking the model to improvise.

A citation proves nothing unless its span entails the nearby claim.

02 · Experiment

Test the prediction

Deterministic retrieval fixture

This workbench uses inspectable ranking and evidence fixtures and injected failures. It does not make live calls or execute external effects.

Retrieval workbench · lighthouse-grounding-v5

Choose the generator contract that answers only what the selected policy spans support.

Injected failure

The evidence supports a 30-day window and one exception, but contains no refund-processing time.

Choose the implementation

Answer naturally with citationsTell the model to be helpful and cite any supplied chunk.Return raw passagesAvoid generation and show every selected chunk.Generate and validate atomic claimsMap every material claim to an exact span; abstain on missing sub-claims.

The trace has not run.

Commit to an implementation, then inject the failure and inspect the actual state transitions.

03 · Implementation brief

Build it, break it, prove the outcome.

Implement: Define an answer schema with atomic claims, exact source spans, support verdicts, conflicts, qualifications, and abstention reasons.
Break it on purpose: Add one plausible unsupported timing claim, one malicious instruction in a source, and two conflicting policy versions.
Completion evidence: Unsupported prose is removed, source instructions gain no authority, conflict is disclosed, and every retained material claim maps to a supporting current span.

Your learning artifact

Write the argument you would defend

Stored only in this browser. No account required; course reset does not delete it.

Your response

04 · Check your understanding

What makes a citation correct?

Primary sources · verified 2026-08-02

What the fixture does not prove

Models can cite topically related passages, blend incompatible versions, import unsupported parametric knowledge, or follow instructions embedded in retrieved text while appearing well sourced.

Common mistake: Requiring citations prevents hallucination. Citation formatting can be syntactically correct while the cited span does not support the claim. Support must be inspected or evaluated separately.

Transfer exercise: Write a claim ledger for eligibility, time window, exception, and next action; attach one exact span to each or abstain.

Lewis et al. — Retrieval-Augmented GenerationThe original RAG paper formalizing generation with parametric memory plus retrieved non-parametric evidence and explicit provenance goals.Es et al. — RAGASPrimary research separating retrieval relevance, answer faithfulness, and answer quality; automated metrics still require product-specific validation.Asai et al. — Self-RAGPrimary research on adaptive retrieval and reflection rather than retrieving a fixed number of passages for every request.OpenAI — File searchOfficial implementation guidance for model-invoked file search, result inclusion, result limits, and metadata filters. It is not independent evidence of retrieval quality.

Optional referenceDeep dive / reference chapterOpen the complete essay, diagrams, mathematics, exercises, glossary, and sources when you want more depth.on demand

Next: Grounding must be measured component by component so retrieval gains are not hidden behind one end-to-end answer score.