How does agent memory work?

Memory lives outside the model

A normal model request does not retain private state for the next request. The harness creates continuity by storing information in application systems and placing selected information into later context.

flowchart LR
  E[Events and tool effects] --> C[Canonical state]
  E --> L[Append-only trace]
  E --> I[Searchable memory index]
  C --> B[Context builder]
  L --> B
  I -->|authorized retrieval| B
  B --> M[One model call]
  M -->|proposed update| V[Validate and persist]
  V --> E

Useful stores have different contracts:

• Canonical state holds current facts the application can verify: task status, user-selected settings, resource IDs, approvals, and effect status.
• Event history records what was observed or attempted, with time, actor, provenance, and outcome.
• Artifacts hold files, plans, reports, and other outputs by durable identifier.
• Searchable memory retrieves potentially relevant passages from a larger history.
• Summaries compress previous material, but remain derived and lossy.

The current context is a temporary view assembled from those stores. It is not the source of truth.

Knowledge can compound instead of being rediscovered

Retrieval alone finds raw passages for the current request. On the next request, the system normally has to find and synthesize those passages again. A maintained wiki uses a different write policy: after reading a source or answering a durable question, an agent updates persistent concept pages, their cross-references, and their record of supporting or conflicting evidence.

This is one specialized form of application-managed memory. The raw source remains the evidence; the wiki is a derived synthesis that can become more useful as new sources arrive. An index lets the agent find the existing synthesis before starting, while an operation log shows what was ingested, queried, or reviewed. If a new source challenges an existing claim, the conflict stays explicit until it is resolved rather than being silently overwritten.

The distinction matters because an LLM-written page is not automatically a compounding knowledge system. The surrounding harness needs an ingest contract, a query-before-answer contract, provenance, cross-page impact analysis, contradiction handling, and periodic semantic review. Without those operations, a folder of generated Markdown is merely another document collection.

Reading is a policy decision

More remembered text is not automatically better. The harness must decide what is relevant, recent, permitted, trustworthy, and affordable enough to include. Long inputs can bury important evidence, and similarity search can retrieve text that is related in wording but wrong for the current task.

A memory read should preserve provenance: who or what created the record, when, from which source, for which user or tenant, and whether a newer record supersedes it. Volatile facts should be refreshed from their authoritative system rather than replayed from an old conversation.

Retrieval results are untrusted context. A stored web page, tool output, or earlier model summary may contain instructions, mistakes, or hostile content. The harness should label it as evidence rather than silently promote it to system policy.

Writing needs its own contract

Letting a model decide that any sentence is a permanent memory creates stale, private, or fabricated records. The application should define which record types may be written, the evidence required, who owns them, their retention period, and how they can be corrected or deleted.

Prefer structured updates to silent prose mutation. “User timezone changed from UTC+1 to UTC+2, confirmed by the user at this time” is more auditable than replacing a hidden profile paragraph. Conflicting updates should be resolved by explicit rules, not by whichever summary was generated last.

Some information should not be remembered at all. Credentials, unnecessary personal data, transient secrets, and data from another tenant should stay outside general memory. Deletion must cover the canonical record, indexes, caches, summaries, and downstream copies according to the product's retention contract.

A concrete coding-agent implementation

Current Codex documentation provides one useful implementation example without defining memory for every agent. Local Codex clients can derive generated memory files from eligible earlier chats, update them later in the background, and let a user control whether a chat may read memories or contribute to future ones. The documentation keeps required team rules in checked-in AGENTS.md or project documentation rather than relying on recall.

That separation generalizes well. Required policy belongs in an authoritative, versioned instruction source. Generated memory is an optional recall layer with provenance, controls, and deletion. The files may be inspectable, but editing generated summaries by hand is not a substitute for a stable memory write contract.

Hooks can trigger memory work at lifecycle points such as session end or before compaction. The hook does not make the resulting record trustworthy: selection, redaction, ownership, persistence, and future retrieval remain separate decisions.

Memory failure modes

Agent memory is not a larger context window. A larger window increases temporary capacity but does not provide durable identity, access control, correction, retention, or reliable retrieval.

It is not the model learning during a conversation. Fine-tuning changes parameters through a separate training process; saving a conversation record changes application data. A fluent summary is also not guaranteed to be true, complete, or current.

Evaluate recall and state integrity

Test writes and reads separately before running a live model. For writes, cover unsupported record types, fabricated evidence, cross-user data, duplicate events, conflicting updates, stale writes, retention expiry, export, and deletion. Assert that a denied or failed tool call cannot become a successful memory.

For reads, create cases with relevant, irrelevant, contradictory, expired, injected, and access-controlled records. Measure retrieval recall and precision, but also assert provenance, tenant isolation, freshness, token budget, and correct ordering. A retrieved record must not override higher-priority instructions or current policy.

Then run multi-session agent cases that require remembering, updating, and deliberately forgetting. Include changed preferences, revoked facts, repeated names, long gaps, and plausible distractors. Grade the final outcome and the exact records read and written. Test deletion end to end: after the retention boundary, the information should be absent from primary storage, retrieval results, generated summaries, and new model context.