Memory & State

January 30, 2026

LLMs are stateless. Every prompt starts fresh — no memory of previous conversations, no accumulated knowledge, no persistent identity. Memory systems give agents continuity, turning one-shot interactions into ongoing relationships.

The Memory Problem

As one researcher observed:

Most agents shine in-session, then restart from zero. No persistent identity = no true long-horizon growth.

This creates a philosophical question: if an agent restarts without memory, is it still the same agent?

Memory Types

Agent memory operates on multiple timescales:

Short-Term Memory

The conversation history. Everything said in the current session. This provides coherence within a single interaction but vanishes when the session ends.

Characteristics:

Working Memory

Active information the agent is using right now. Like a human's mental scratchpad: task context, intermediate results, current goals.

Implementations:

Long-Term Memory

Persisted across sessions. The agent "remembers" things learned previously. This is where engineering gets interesting.

Implementations:

Memory Architectures

Different approaches to long-term memory:

Vector Store + RAG

Store memories as embeddings. Retrieve semantically similar content when relevant.

User: "What was that restaurant I liked?"
       ↓ embed query
       ↓ search vector store
       ↓ retrieve: "Loved dinner at Flour + Water on 3/15"
       ↓ inject into context
Agent: "You enjoyed Flour + Water back in March!"

Pros: Scalable, finds semantic matches Cons: May miss exact matches, retrieval isn't guaranteed

File-Based Memory

Store memories as files. Let the agent read/write explicitly.

/memory/
  2026-01-30.md    # Daily notes
  MEMORY.md        # Long-term curated memories
  preferences.json # Structured data

Pros: Simple, inspectable, agent-controlled Cons: Manual curation needed, less automatic

Knowledge Graphs

Store memories as relationships between entities.

(User) --[prefers]--> (Italian food)
(User) --[visited]--> (Flour + Water)
(Flour + Water) --[serves]--> (Italian food)

Pros: Rich relational queries, explicit connections Cons: Complex to build, requires schema

The State Problem

Beyond memory, agents need state: where they are in a workflow, what tools they've used, what's been decided.

State components:

As a developer noted:

Context management is the single most important thing you can work on when building AI agents (including agent memory, available tools, even newer concepts such as TO-DOs and skills). And the reason is simple: LLMs are stateless probabilistic functions!

Memory Best Practices

1. Don't Trust "Mental Notes"

Agents can't actually remember between turns unless you persist it.

"Mental notes" don't survive session restarts. Files do. When someone says "remember this" → write it to a file.

2. Separate Memory Layers

3. Memory Maintenance

Periodically review and consolidate:

4. Security Boundaries

Memory persistence creates security implications:

Framework Support

Different frameworks handle memory differently:

The Identity Question

If an agent's memory is its identity, what happens when:

These are open questions without clear answers. Current practice: treat memory as context that shapes behavior, not as true identity.

Sources

See also: Context Loading · Tools & Function Calling · Orchestration