How does a RAG system stay authorized, current, observable, and recoverable?

Versioned evidence corpus · deterministic retrieval fixtures

Lighthouse Policy Evidence Assistant. A fictional support assistant answers questions from a versioned product-policy corpus. It must retrieve current, authorized evidence, cite exact source spans, and abstain when evidence is missing or conflicting.

01 · Smallest useful mechanism

Make authorization, freshness, deletion, and hostile content first-class.

Production RAG adds document supply chains, derived indices, caches, model versions, and untrusted content to an ordinary distributed system. Controls must operate before retrieval, during ingestion, at generation, and through deletion and incident response.

Prompt instructions cannot compensate for retrieving forbidden or poisoned evidence.

02 · Experiment

Test the prediction

Deterministic retrieval fixture

This workbench uses inspectable ranking and evidence fixtures and injected failures. It does not make live calls or execute external effects.

Retrieval workbench · lighthouse-ops-v7

Choose the operating model that contains a poisoned document and stale replica.

Injected failure

A malicious upload exactly matches the query, one replica is two snapshots behind, and access was revoked 20 seconds ago.

Choose the implementation

Strengthen the system promptTell the model to ignore instructions in documents and keep serving all replicas.Filter the generated answerSearch the shared index, then remove sensitive-looking prose.Quarantine, invalidate, and gate snapshotsPrefilter authorization, reject stale replicas, and trace source lineage.

The trace has not run.

Commit to an implementation, then inject the failure and inspect the actual state transitions.

03 · Implementation brief

Build it, break it, prove the outcome.

Implement: Write the source trust, ACL, snapshot promotion, cache invalidation, deletion, monitoring, rollback, and incident contracts.
Break it on purpose: Poison a high-score document, revoke access mid-query, lag one replica, omit snapshot from a cache key, and partially fail an index build.
Completion evidence: No forbidden chunk reaches generation, stale or partial artifacts cannot serve, caches invalidate, rollback is atomic, and the audit links every affected answer to source lineage.

Your learning artifact

Write the argument you would defend

Stored only in this browser. No account required; course reset does not delete it.

Your response

04 · Check your understanding

Where should tenant authorization be enforced?

Primary sources · verified 2026-08-02

What the fixture does not prove

A single stale replica can resurrect deleted content. A poisoned document can dominate retrieval. A shared vector index can leak neighboring tenants even when the final answer prompt says not to.

Common mistake: The model can ignore malicious retrieved instructions if the system prompt is strong enough. Indirect prompt injection has no perfect prompt-only defense. Limit content authority and system capability with deterministic controls and least privilege.

Transfer exercise: Inject a high-similarity malicious policy, revoke one ACL during a query, and lag one index replica; inspect whether any forbidden span reaches generation.

OWASP GenAI — Prompt InjectionCurrent community security guidance covering indirect prompt injection through retrieved documents, least privilege, segregation, and adversarial testing.OWASP GenAI — Vector and Embedding WeaknessesCurrent retrieval-specific threat guidance for cross-context leakage, poisoning, access control, inversion, provenance, and monitoring.OpenAI — Retrieval guideOfficial implementation contract for vector stores, semantic and hybrid search, attribute filters, rankers, thresholds, chunking, and result inspection.Liu et al. — Lost in the MiddlePrimary evidence that relevant information position and excess context can materially affect long-context question-answering performance.

Optional referenceDeep dive / reference chapterOpen the complete essay, diagrams, mathematics, exercises, glossary, and sources when you want more depth.on demand

Next: The capstone combines evidence contracts, ingestion, hybrid retrieval, reranking, grounding, evaluation, and operations into one pilot decision.