Memory: Turning an Agent from a "Tool" into an "Assistant"
An Agent without memory starts from scratch every conversation. You tell it your name, your job, your preferred learning style — next time, it has no idea who you are.
That's not a bug. It's a missing architectural layer.
LLMs are stateless by design. Every call is independent. If you want an Agent to remember things, you must explicitly store, manage, and retrieve memory at the architecture level. That's exactly what memory management solves.
This article breaks Agent memory down into four dimensions: a taxonomy of memory types, three context management strategies, LangGraph's two memory primitives (checkpointer and store), and an auto-compression scheme for arbitrarily long conversations.
Four Memory Types: From Cognitive Science to Engineering
Borrowing from cognitive science's model of human memory, Agent memory naturally decomposes into four layers — each with a corresponding LangGraph implementation:
┌──────────────────────────────────────────────────────────────┐
│ Memory Hierarchy │
├──────────────────────┬───────────────────────────────────────┤
│ Sensory Memory │ In-flight messages for the current │
│ │ turn. Lifetime: one LLM call. │
├──────────────────────┼───────────────────────────────────────┤
│ Working Memory │ Recent message history (last K turns) │
│ │ Implementation: inject messages list │
│ │ into prompt │
├──────────────────────┼───────────────────────────────────────┤
│ Episodic Memory │ Vectorized / summarized history │
│ │ Implementation: summary compression + │
│ │ VectorStore retrieval │
├──────────────────────┼───────────────────────────────────────┤
│ Semantic Memory │ Long-term user preferences and facts │
│ │ Implementation: LangGraph store │
│ │ (KV Store) │
└──────────────────────┴───────────────────────────────────────┘
Sensory Memory: The Current Turn
The most transient layer. The input and output of one LLM call — used and discarded:
q = "What is len([1, 2, 3])?"
answer = llm.invoke([HumanMessage(q)])
# answer.content → "len([1, 2, 3]) equals 3."
# Once this invoke() returns, the answer is gone
There's nothing to "manage" here — sensory memory is just the LLM call itself.
Working Memory: Bounded Conversation History
Prepend the last few turns of conversation into the prompt. The effect is immediate and obvious:
history = [
HumanMessage("My name is Li Lei, I'm a Python engineer"),
AIMessage("Hello, Li Lei! Nice to meet you."),
HumanMessage("I've been learning LangGraph lately"),
AIMessage("LangGraph is powerful — great for building stateful Agents."),
]
test_q = "What's my name again?"
Measured output:
With history → "Yes, you told me your name is Li Lei, and you're a Python engineer..."
Without history → "I'm sorry, I cannot recall the name you told me earlier, as an AI
I don't have persistent memory to store personal data..."
The gap is stark. The drawback: token cost grows linearly with conversation length, so you need truncation or summarization to keep it bounded.
Episodic Memory: Compressed History Snippets
When conversation history grows long, stuffing everything into the prompt becomes expensive. Episodic memory's approach: compress first, then store:
long_history = history * 4 # 16 messages
summary = llm.invoke([
SystemMessage("Compress the following conversation into under 60 words, preserving key facts"),
HumanMessage(str([m.content for m in long_history])),
])
# → "Li Lei, Python engineer, learning LangGraph enthusiastically,
# praised it as powerful for stateful Agents."
16 messages compressed to 28 words. Next turn, use the summary instead of raw history — token cost drops dramatically.
Semantic Memory: Cross-Session User Facts
The most persistent layer. Survives across conversations, specifically for storing long-term facts about the user (name, role, preferences):
# Store user profile in KV store — readable in any future session
user_profile = {
"name": "Li Lei",
"role": "Python engineer",
"interests": ["LangGraph", "Agent development"],
"level": "intermediate",
}
# With this, the Agent gives personalized answers
# "What should I learn next?" → recommend LangGraph advanced patterns,
# not Python basics
Three Context Management Strategies: Truncation / Summarization / Retrieval
When conversation history keeps growing and the context window can't keep up, you have three options:
Strategy 1: Truncation
The simplest approach — keep only the last N messages, discard the rest:
# Keep only the last 4 messages
truncated = history[-4:]
resp = llm.invoke(truncated + [HumanMessage(test_q)])
Tested with 8-topic conversation history (16 messages), truncated to last 4, then asked "What is a Python list?":
Earliest visible message after truncation: "Explain Python decorators" (topic 5 — "lists" was topic 1)
Answer: "A Python list is a built-in data structure..." (answered from LLM's own knowledge)
⚠ Lost the fact that we "covered" lists — LLM answers from general knowledge only
Best for: Scenarios where historical continuity doesn't matter, or pure Q&A Agents where past context is irrelevant.
Strategy 2: Summarization
Use the LLM to compress long history into a summary paragraph, then use the summary instead of raw history for subsequent turns:
summary_resp = llm.invoke([
SystemMessage("Compress conversation history into a summary (≤80 words), keep all topic names"),
HumanMessage("\n".join([f"{m.type}: {m.content}" for m in history])),
])
# → "Python lists are mutable and ordered, tuples are immutable and memory-efficient,
# dicts map keys to values, sets store unique elements,
# functions encapsulate logic, classes enable OOP,
# decorators wrap functions, generators enable lazy evaluation."
# 16 messages → 66-word summary
With the same "What is a Python list?" question, the summarization approach answers with awareness that "we've discussed this" — not just generic knowledge.
Strategy comparison:
| Strategy | Token Cost | Information Retained | Complexity | Best For |
|---|---|---|---|---|
| Truncation | Lowest | Recent turns only | Trivial | Q&A, stateless tasks |
| Summarization | Low | Global thread | Medium | Teaching, consulting, long sessions |
| Retrieval | Lowest | Precisely relevant | High (needs vector DB) | Knowledge bases, multi-domain Agents |
Strategy 3: Retrieval
Pull only the history snippets semantically relevant to the current question — the most effective and most complex approach:
# Simplified demo: keyword filter (production uses vector similarity)
relevant = [m for m in history if "list" in m.content.lower()]
# 16 messages → 3 relevant ones
resp = llm.invoke(relevant + [HumanMessage(test_q)])
Best for: Knowledge-base Agents, personal assistants with extensive user histories.
LangGraph checkpointer: Within-Session State Persistence
LangGraph's MemorySaver (checkpointer) uses thread_id to distinguish sessions and automatically accumulates conversation history within a session:
from langgraph.checkpoint.memory import MemorySaver
from langgraph.prebuilt import create_react_agent
checkpointer = MemorySaver()
agent = create_react_agent(model=llm, tools=[get_weather], checkpointer=checkpointer)
# Same thread_id = same session, state persists across turns
config_a = {"configurable": {"thread_id": "weather_001"}}
Cross-Turn Reference in Practice
Three consecutive weather queries in the same session:
[Turn 1] User: What's the weather in Beijing today?
Agent: Beijing today: sunny, 25°C, NE wind level 3, good air quality.
(Messages in state: 4)
[Turn 2] User: What about Shanghai? ← "What about" has no explicit referent
Agent: Shanghai today: overcast, 22°C, SE wind level 2, light smog.
(Messages in state: 8)
[Turn 3] User: Which city is better for going out today? ← needs both prior results
Agent: Given Shanghai's smog, Beijing is the better choice today.
(Messages in state: 10)
Turns 2 and 3 both depend on prior history — checkpointer handles cross-turn context automatically.
Session Isolation
Different thread_id values are completely independent:
[New session — thread_id: weather_002]
User: What city did I just ask about?
Agent: You asked "what city," but no specific city name was provided.
If you'd like to check the weather somewhere, please tell me the city.
→ New thread_id has no history — has no idea what was asked before
Session Continuation
Return to the same thread_id later — history is still there:
[Session A continued — same thread_id]
User: Compare those two cities with Shenzhen
Agent: Shenzhen today: showers, 27°C, SW wind level 2, thunderstorm warning.
(Provided three-city comparison: Beijing / Shanghai / Shenzhen)
→ Remembered the previous Beijing and Shanghai queries
MemorySaver is in-memory — data is lost on process restart. For production, use SqliteSaver (local file) or PostgresSaver (database).
LangGraph InMemoryStore: Cross-Session Long-Term Memory
The checkpointer handles memory within a single session. Cross-session long-term memory requires the store:
checkpointer → bound to thread_id, valid for the session's lifetime
store → bound to user_id, persists across all sessions
Core API
from langgraph.store.memory import InMemoryStore
store = InMemoryStore()
# Write: (namespace, key, value)
store.put(("user_facts", user_id), key, {"fact": "Li Lei, backend engineer"})
# Read: search all entries under a namespace
facts = store.search(("user_facts", user_id))
for item in facts:
print(item.value["fact"])
# Precise read
item = store.get(("user_facts", user_id), specific_key)
Cross-Session Memory in Practice
In Session A, the Agent automatically extracts and stores user information from the conversation:
[Session A] User said three things → auto-extracted and stored:
• Li Lei, backend engineer
• Python, Go, LangGraph, Agent development
• Prefers hands-on practice over reading docs
In a completely new Session B (new thread_id, same user_id), asking "Do you know me?":
[Session B — brand new thread_id]
User: Hi, do you know me?
Agent: Hello! Based on your profile, yes I do. You're Li Lei, a backend engineer
skilled in Python, Go, LangGraph, and Agent development.
You prefer hands-on practice over reading documentation.
How can I help you today?
→ Brand new thread_id, but store data persists across sessions
Data for different user_id values is fully isolated and cannot bleed across users.
checkpointer vs store
# Short-term memory: checkpointer — bound to thread_id, valid within session
app = graph.compile(checkpointer=MemorySaver())
result = app.invoke(input, config={"configurable": {"thread_id": "abc"}})
# Long-term memory: store — bound to user_id, valid across sessions
store = InMemoryStore()
app = graph.compile(store=store, checkpointer=MemorySaver())
# Operate on store inside a node
store.put(("user_facts", user_id), key, {"fact": "..."})
stored = store.search(("user_facts", user_id))
In production, swap InMemoryStore for PostgresStore or RedisStore to get real persistence — the architecture stays identical.
Auto-Summarization: RemoveMessage + Summary Rotation
When message count exceeds a threshold, trigger automatic compression — this is the core mechanism that keeps an Agent coherent over infinitely long conversations.
Graph Structure
[chat node]
│
├─ message count ≤ threshold → END
└─ message count > threshold → [compress node] → END
The compress Node: Deleting Old Messages with RemoveMessage
def compress_node(state: SummaryState) -> dict:
messages = state["messages"]
to_compress = messages[:-2] # keep the 2 most recent, compress the rest
keep = messages[-2:]
# old messages → new summary
new_summary = llm.invoke([
SystemMessage("Compress the following into a summary under 120 words"),
HumanMessage(existing_summary + old_messages_text),
]).content
# RemoveMessage: tells the add_messages reducer to delete these messages
remove_ops = [RemoveMessage(id=m.id) for m in to_compress]
return {"messages": remove_ops, "summary": new_summary}
RemoveMessage is LangGraph's dedicated message-deletion operator. When the add_messages reducer encounters it, it removes the matching message ID from state — rather than appending.
Measured Results
11 turns of conversation, compression threshold at 8 messages:
[Turns 1–4] Message count: 2/4/6/8 | Summary: ○ none
[Compression triggered] 10 messages → compress 8, keep 2
[New summary] Python list common methods include search, sort, add, remove.
dict.get() avoids KeyError, returns default value.
*args accepts variable positional args, **kwargs accepts keyword args...
[Turn 5] Message count: 2 | Summary: ✓ compressed ← restarts from 2 after compression
[Compression triggered again] 10 messages → compress 8, keep 2
[Turn 11] Final summary:
"Based on our discussion, here's a summary of Python concepts covered:
1. Python list comprehensions...
2. Set comprehensions... ← inherited from Turn 1 via summary chain
3. Lambda functions..."
Key result: All 11 turns kept only 2–8 active messages — but through the summary chain, every piece of knowledge from turn 1 was preserved at turn 11.
State Design
class SummaryState(TypedDict):
messages: Annotated[list, add_messages] # add_messages handles RemoveMessage
summary: Optional[str] # accumulated history summary, injected into system prompt
def chat_node(state: SummaryState) -> dict:
summary = state.get("summary") or ""
system_parts = ["You are a helpful assistant."]
if summary:
system_parts.append(f"\n\n[Conversation History Summary]\n{summary}")
resp = llm.invoke([SystemMessage("".join(system_parts))] + state["messages"])
return {"messages": [resp]}
Memory Management Design Checklist
Everything to consider when building a complete Agent memory system:
Short-term memory (checkpointer)
- [ ] Choose the right checkpointer backend (MemorySaver for dev, SqliteSaver/PostgresSaver for prod)
- [ ] Assign each user/session a unique
thread_id - [ ] Set a history truncation threshold to prevent unbounded token growth
Long-term memory (store)
- [ ] Organize user data by namespace:
(type, user_id)e.g.,("user_facts", uid) - [ ] Apply confidence filtering during extraction — avoid storing meaningless noise
- [ ] Replace
InMemoryStorewithPostgresStore/RedisStorein production
Context compression
- [ ] Determine compression threshold (8–20 messages is a reasonable starting range)
- [ ] Make the summary prompt explicit about what to preserve (topic names, key decisions, user preferences)
- [ ] Test the summary chain: does the Turn-N summary still carry information from Turn 1?
- [ ] Use
RemoveMessagerather than replacing the entire messages list (the latter breaks with checkpointer)
Memory read/write timing
- [ ] Read memory: at the start of chat_node, inject into system prompt
- [ ] Write memory: at the end of chat_node, extract new user information
- [ ] Avoid writing on every turn (set confidence or content-length filters)
Summary
Five core takeaways:
- Four memory types, each with a distinct role: Sensory memory is the LLM call itself; working memory is conversation history; episodic memory is compressed summaries; semantic memory is a cross-session KV store
-
checkpointer manages sessions, store manages users:
thread_idis the session dimension,user_idis the user dimension — keep them separate - Summary compression is the key to long conversations: RemoveMessage + summary injection keeps token cost bounded while preserving all prior knowledge
-
Session isolation is non-negotiable: Different
thread_idhistories never bleed; differentuser_idlong-term memories never bleed - InMemoryStore → PostgresStore is a one-line swap: Architecture stays constant, backend is pluggable
Up next: Knowledge Base Integration — the real difference between RAG-as-a-tool and a RAG pipeline, multi-knowledge-base routing, and how an Agent decides when to retrieve, what to retrieve, and how many times.
References
- LangGraph Memory Concepts
- LangGraph Persistence (checkpointer)
- LangGraph Store (cross-session memory)
- Full demo code for this series: agent-05-memory-system
Find more useful knowledge and interesting products on my Homepage
