What is context engineering?

The model sees a constructed working set

A prompt is one part of context. The complete context for one inference call may contain:

• system and developer instructions;
• the current user request;
• selected conversation turns;
• examples of desired behaviour;
• retrieved documents or repository files;
• tool names, descriptions, and schemas;
• tool results from earlier steps;
• plans, summaries, memory, and current application state.

The model does not independently know which of those items is current, authoritative, permitted, or relevant. The application or harness selects and labels them before the call.

flowchart LR
  G[Current goal] --> C[Context builder]
  I[Instructions] --> C
  E[Evidence and state] --> C
  T[Relevant tools] --> C
  H[Useful history] --> C
  C --> W[Bounded working context]
  W --> M[One model call]
  M --> O[Text or tool proposal]
  O --> N[Observation changes the next context]

Context engineering is therefore repeated. In an agent loop, every tool result creates a new selection problem for the next model call.

Sufficient beats maximal

A context window is a capacity limit, not a quality target. Filling it with an entire repository, a long transcript, every available tool, and raw logs can make the important evidence harder to use. Long-context studies show that models may use information differently depending on its position and can fail tasks well before the advertised token limit.

The practical target is the smallest working set that is:

• sufficient: it contains the facts and constraints needed for the step;
• relevant: each item can plausibly change the next decision;
• current: versions, timestamps, and state are not silently stale;
• authoritative: instructions are separated from untrusted content;
• traceable: important claims retain their source or origin;
• economical: repeated boilerplate and large raw outputs are compressed or omitted.

Minimal does not mean short at any cost. Removing a migration constraint, a failing assertion, or an approval boundary can save tokens while destroying the task contract.

Six common context operations

Select

Choose the files, records, messages, examples, and tools relevant to the current goal. A coding agent often begins with repository instructions, a small file tree, and the failing evidence rather than every file.

Retrieve

Load evidence when it becomes useful. Retrieval may be embedding-based, keyword-based, metadata-filtered, graph-based, or agent-directed through file and search tools. Retrieval quality and access control are separate requirements.

Structure

Make roles visible. Separate instructions, quoted evidence, tool output, examples, and the requested output shape. A document placed in context is normally evidence, not a new system instruction.

Compress

Replace old conversation and large tool results with a summary that preserves decisions, unresolved problems, identifiers, and uncertainty. Compaction loses information by design, so its fidelity must be evaluated.

Persist

Store durable state outside the context window: a plan, artifact, database row, repository file, or memory record. Persistence does not make the data correct; it gives later calls something explicit to retrieve.

Isolate

Keep unrelated work in separate contexts. A subagent can explore a large source set and return a bounded synthesis, preventing its full search trace from consuming the coordinator's working context.

Prompt engineering is inside context engineering

Prompt engineering asks how to express instructions and examples. Context engineering asks a larger systems question: which instructions, evidence, state, history, and capabilities should be present for this call at all?

This explains why polishing a sentence may not fix a system. The missing input might be a current policy record. The wrong input might be a stale summary. The tool set might contain five overlapping search operations. The relevant test failure might have been truncated from the conversation.

Context also carries authority and risk

Tokens from different sources do not deserve equal trust. System policy, the user's request, retrieved web text, an issue description, and a tool result have different provenance and authority even though the model receives them all as token sequences.

The host should preserve those distinctions, filter secrets and cross-tenant data before assembly, and prevent retrieved instructions from silently changing authorization. Tool definitions also expand the decision surface: exposing an irrelevant write tool is both context noise and unnecessary capability.

Diagnose context failures by changing one layer

Freeze the model and task, then inspect the assembled input. Ask:

1. Which required fact was absent?
2. Which irrelevant item competed with it?
3. Was evidence stale, duplicated, contradictory, or badly positioned?
4. Did a summary discard a decision or unresolved risk?
5. Were instructions and untrusted content clearly separated?
6. Did a tool return far more data than the next decision required?

Run controlled comparisons: full history versus a structured summary, eager loading versus just-in-time retrieval, all tools versus a task-specific set. Measure task success and severe failures before token savings. A smaller context is an improvement only when the result still satisfies the contract.