Issue 2026-11-18 · Industry · 开源 · 研究 · 模型发布

Running 180B MoE inference in 32GB of RAM with Transformers

Running 180B MoE inference in 32GB of RAM with Transformers

A Hugging Face blog post shows that streaming MoE experts from SSD lets a 180B-parameter model run in 32GB of RAM with Transformers, reaching token generation within 6% of Strata. Since experts make up over 90% of MoE weights, keeping only hot experts in VRAM enables large models on consumer hardware.

Hugging Face Blog2 d ago
Read original ↗