OpenClaw Memory

Part 1: The 4 Layers

Understanding how memory actually works

What is Memory?

Most people think of memory as one thing:

"Does my agent remember or not?"

But it's actually 4 different systems working together.

Each layer has different characteristics, different lifespans, and different failure modes.

The 4 Memory Layers

  1. Bootstrap Files - Permanent identity
  2. Session Transcript - Conversation history
  3. Context Window - Active working memory
  4. Retrieval Index - Searchable archive

Layer 1: Bootstrap Files

The permanent identity layer

These files are loaded from disk at every session start:

  • SOUL.md - Personality, tone, boundaries
  • AGENTS.md - Workflow rules, tool conventions
  • USER.md - Information about you
  • MEMORY.md - Current working state
  • TOOLS.md - Available tools and APIs

Why Bootstrap Files Survive

Key characteristic: Reloaded from disk

  • Not stored in conversation history
  • Read fresh from filesystem at session start
  • Immune to compaction (they're outside the transcript)
  • Changes take effect immediately on next session

This is your most durable layer

Bootstrap Files: Limits

Default limits:

  • 20,000 characters per file
  • 150,000 characters total

If a file exceeds the limit:

  • It gets truncated
  • The truncated portion is invisible to the agent
  • Check with /context list command

Bootstrap Files: What Goes Where

  • SOUL.md - Character traits, speaking style, values
  • AGENTS.md - Process rules, "always do X before Y"
  • USER.md - Facts about the user that rarely change
  • MEMORY.md - Current projects, active context (agent can edit)
  • TOOLS.md - API endpoints, credentials references, tool usage notes

Rule: Only you edit SOUL/AGENTS/USER/TOOLS. Agent only edits MEMORY.md.

Layer 2: Session Transcript

The conversation history layer

Every conversation is saved as a file on disk:

  • Stored in ~/.openclaw/sessions/
  • Contains full message history (user + assistant + tool calls)
  • Rebuilt into context when you continue a session
  • Persists across restarts

Session Transcript: The Problem

What happens during compaction:

  1. Context window approaches limit
  2. OpenClaw triggers compaction
  3. Old messages are summarized into a compact form
  4. Summary replaces detailed history in context
  5. Model can no longer see original messages

Important: The raw transcript file still exists on disk, but the model only sees the summary.

Session Transcript: Lifespan

  • Before compaction: Full detailed history available
  • After compaction: Only summary + recent messages
  • Recent messages: Last ~20,000 tokens stay intact
  • Old messages: Compressed into summary

This is why instructions given in chat don't survive long sessions

Session Transcript: What Gets Lost

During compaction, these are at risk:

  • Exact wording of earlier instructions
  • Nuance and context from old messages
  • Specific constraints mentioned mid-conversation
  • Preferences stated casually
  • Images from earlier in the session

What survives: Anything written to bootstrap files

Layer 3: Context Window

The active working memory layer

This is the fixed-size container where everything competes for space:

  • System prompt (instructions from OpenClaw)
  • Bootstrap files (SOUL.md, AGENTS.md, etc.)
  • Conversation history (or its summary)
  • Tool calls and their results
  • Current message being processed

Context Window: Size Matters

Typical sizes:

  • Claude Opus/Sonnet: 200,000 tokens
  • GPT-4: 128,000 tokens
  • Gemini Pro: 1,000,000+ tokens

1 token ≈ 0.75 words (English)

200K tokens ≈ 150,000 words ≈ 300 pages

Sounds like a lot, but fills up faster than you think.

Context Window: What Fills It

Large consumers:

  • Tool results - File reads, web snapshots, API responses
  • Long conversations - Multi-turn back-and-forth
  • Code blocks - Full file contents in messages
  • Bootstrap files - Loaded every turn

When it fills → compaction fires

Context Window: The Trigger

Compaction triggers when:

Current usage reaches:

context_limit - reserveTokensFloor - softThreshold

Example with 200K context:

200,000 - 40,000 - 4,000 = 156,000 tokens

At 156K tokens, compaction fires to free up space.

Layer 4: Retrieval Index

The searchable archive layer

A searchable index that sits beside your memory files:

  • Indexes MEMORY.md and memory/ directory
  • Agent queries with memory_search tool
  • Returns relevant snippets from past sessions
  • Uses hybrid search (keyword + semantic)

Retrieval Index: How It Works

The process:

  1. You write information to memory files
  2. OpenClaw indexes the content
  3. Agent searches with memory_search("pricing decision")
  4. Index returns relevant snippets with file paths
  5. Agent reads full context with memory_get

Retrieval Index: The Catch

Critical limitation:

Only works if information was written to files first

  • Chat messages are NOT indexed
  • Tool results are NOT indexed
  • Only markdown files in workspace are indexed

This is why "write it to a file" is so important.

Retrieval Index: Hybrid Search

Keyword search: Finds exact words

  • Fast and precise
  • Misses synonyms and related concepts

Semantic search: Finds meaning

  • Understands "pricing decision" = "we picked the $29 tier"
  • Uses embeddings to capture meaning

Hybrid = both working together

The 4 Layers: Summary

  1. Bootstrap Files - Permanent, reloaded from disk, immune to compaction
  2. Session Transcript - Full history on disk, but gets summarized in context
  3. Context Window - Fixed size, everything competes, triggers compaction when full
  4. Retrieval Index - Searchable archive, only works for files

Why This Matters

Understanding these 4 layers helps you:

  • Know where to put important information (Layer 1)
  • Understand why chat instructions disappear (Layer 2)
  • Manage context usage effectively (Layer 3)
  • Make information searchable (Layer 4)

Next video: Why these layers fail and how agents forget