AI Memory
How AI agents store and retrieve information across turns, sessions, and tasks.
Unlike humans, LLMs have no persistent memory — each API call starts fresh. "Memory" in AI agents is engineered at the application layer using one or more of these mechanisms:
In-context memory (short-term): Everything in the current context window. Simple but limited — evicted when the window fills up. Cheapest and most reliable.
External memory (long-term):
- Vector store: Past interactions, summaries, or documents embedded and stored in a vector database. Relevant memories are retrieved into context at query time (essentially RAG over the agent's history).
- Key-value store: Structured facts the agent explicitly writes and reads: user preferences, task state, past decisions.
- Database: For structured data the agent needs to query with precision — customer records, inventory, calendar.
Episodic memory: A log of past conversations or actions that can be summarised and retrieved. Platforms like MemGPT, Mem0, and Zep automate this.
Memory management challenges: What to forget, how to summarise long histories without losing key facts, preventing stale memories from misleading current decisions, and keeping memory retrieval fast.
In plain terms
A human brain (fast, limited, contextual) paired with a filing cabinet (large, organised, requires lookup). Most agent memory architectures combine both — fast context for current work, external store for accumulated history.
Related concepts
AI Agents
AI that plans and takes actions without you guiding every step.
Retrieval-Augmented Generation
Giving the AI access to your documents before it answers.
Vector Database
A database that stores meaning, not just text — finding similar content by concept rather than keyword.
Context Window
How much text an AI can see and remember at once.