Issue 2026-11-07 · Industry · 开源 · 研究

llama.cpp PR runs missed MoE experts on CPU

llama.cpp pull request #30966 adds llama_moe_cache, which runs missed MoE experts on the CPU, plus a tuning script. It is seen as a significant speedup for low-VRAM users offloading MoE models.

r/LocalLLaMA3 d ago
Read original ↗