Day 3 · Lesson 3 — Context Engineering: Sessions & Memory
RAG makes an agent an expert on facts. Memory makes it an expert on you.
Memory and RAG both retrieve information into context, so they get conflated constantly. They're complementary rather than competing, and knowing exactly how they differ is the difference between an agent that quotes documents and one that actually knows its user.
From Lesson 2: what turns a raw session transcript into durable knowledge?
Extraction, distilling the meaningful signal from the noisy transcript. That's memory generation, and it's what makes memory ≠ session ≠ RAG.
A memory is a snapshot of extracted, meaningful information, persisted across sessions. RAG retrieves from a static external knowledge base. The paper's crisp framing:1
RAG = the research librarian.
Memory = the personal assistant.
The librarian knows the world's facts from a vast, shared, authoritative library, but nothing personal about you. The assistant carries a private notebook on you: preferences, past conversations, evolving goals. A truly intelligent agent needs both.
| — | RAG | Memory |
|---|---|---|
| Makes the agent expert on | Facts / the world | The user |
| Data source | Static, pre-indexed docs | The user↔agent dialogue |
| Isolation | Shared, global, read-only | Highly isolated, per-user |
| Nature | Static, authoritative | Dynamic, uncertain |
| Write pattern | Batch (offline indexing) | Event-based (per turn / session) |
| Prep | Chunk & embed | Extract & consolidate |
The row that carries the governance weight is isolation: RAG is shared by design; memory is per-user by necessity, because it's derived from personal conversation.
Cutting across all of it, a distinction from cognitive science:1
A memory itself is just content + metadata: the extracted fact (structured
{"seat":"window"} or unstructured prose) plus its owner, source, and labels.
Under the hood, memories live in vector databases (semantic similarity), knowledge graphs (entities + relationships), or a hybrid; and they're scoped to a user (most common), a session (for compaction), or the whole application (shared, must be sanitised).
The isolation row is the governance story. RAG's knowledge base is shared and read-only, a low personal-data risk. Memory is derived from a user's conversation, so it's almost always scoped per-user, and must be: a C Compliance and I Identity requirement. The dangerous exception is application-level memory (shared across users, e.g. procedural playbooks): it must be sanitised of personal content, or one user's data leaks into another's context.
Ladder read: knowing which memories are user-scoped vs app-scoped, and enforcing that boundary, is an L3 Controlled data-governance decision. Get the scope wrong and you've built a cross-user exfiltration channel (Lesson 5).
Ask of every fact: is this world knowledge (RAG) or user knowledge (memory)? Putting user preferences in RAG or product specs in memory is a category error that costs you later.
Standardise a memory schema (content + metadata, with owner + source) so any agent on the team can read the shared store, the framework-agnostic layer from Lesson 2.
Govern memory scope as policy: user-scoped by default, app-scoped only when sanitised. This is the boundary that keeps a personalization feature from becoming a privacy incident.
Recall, don't re-read.
RAG makes an agent an expert on facts; memory makes it an expert on —
The user. RAG = research librarian (shared world facts); memory = personal assistant (a private, per-user notebook of preferences and history).
Compared with RAG, memory is almost always —
Per-user isolated, since it's derived from personal conversation. RAG's base is shared, global, authoritative. That isolation difference is the whole governance story.
A memory of "the right sequence of tool calls to book a trip" is —
Procedural: it's a workflow, a "how." Declarative memory is facts/preferences, the "what." Most commercial memory systems only do declarative well.
List what one of your agents needs to "remember." Sort each item into RAG (world fact, shareable) or memory (about the user, isolated). The mis-sorted ones are where your architecture, and your privacy exposure, will surprise you.
Context Engineering (Day 3), "Memory" pp.27–40, esp. the RAG-vs-memory comparison (Table 1, p.31) and the memory-type taxonomy.
Up next → Day 3 · Lesson 4: Memory Generation — the LLM-driven ETL that turns a noisy transcript into curated knowledge: extraction, the self-editing consolidation step, provenance, and forgetting.
Related: ← Day 3 L2: Sessions · Glossary · Course home