ModelRefs / Memory Systems — Tutorial
Memory Systems — Tutorial
Short-term, long-term, episodic, and semantic memory — how agents remember across sessions. Covers The four types of agent memory, Episodic vs semantic memory.
Overview
Short-term, long-term, episodic, and semantic memory — how agents remember across sessions
Level: Expert. Estimated reading time: 35 minutes.
The four types of agent memory
Human memory is multi-layered. AI agent memory architectures mirror this:
In-context memory (working memory): the current conversation in the context window. Immediate and precise, but bounded by the context limit and reset on every session.
External storage (long-term memory): a database outside the model. Persists across sessions. Agents query it explicitly (retrieve-then-inject).
In-weights memory (parametric knowledge): facts encoded in the model's weights during pretraining. Implicit, always available, but static.
In-cache memory (cached computation): KV cache from previous token sequences. Reduces recomputation cost for repeated prefixes. Anthropic's prompt caching API stores up to 1M tokens.
Most production agents combine all four: in-context for recent history, external storage for long-term facts, in-weights for general knowledge, and cache for cost efficiency.
Episodic vs semantic memory
Episodic memory stores specific events: "On 2025-03-15, the user said they prefer Python over JavaScript." Retrieved by recency or semantic similarity.
Semantic memory stores general facts: "The user is a senior backend engineer who uses Python 3.12 and dislikes verbose comments." Retrieved by relevance to the current task.
Implementation pattern: 1. After each conversation, extract memory-worthy facts: "What should I remember from this conversation?" 2. Embed and store in a vector database with metadata (timestamp, session ID, topic). 3. At the start of each new session, retrieve top-k memories relevant to the current query. 4. Inject into the system prompt: "Context from previous sessions: ..."
The extraction step is important — storing full conversations wastes space and retrieves too much noise. Extracting 3–5 facts per session keeps the store clean.
Memory management and forgetting
Memory stores grow over time and require management:
Deduplication: new memories that contradict or duplicate existing ones should update rather than create a second entry.
Decay / importance weighting: frequently-accessed, recently-relevant memories should persist. Old, rarely-accessed memories should decay. A simple formula: score = importance × e^(-λ × age_days).
Summarisation: periodically summarise a cluster of related memories into a single higher-level memory. "User has asked about Python type hints 12 times" → "User is working on a large Python project focused on type safety."
Privacy: users should be able to inspect and delete their stored memories. Implement memory.list() and memory.delete(id). Don't store PII in embeddings without encryption.
Continue your research
Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Memory Systems — Tutorial.