Issue 2026-11-09 · Industry · 研究 · 开源

HOPE: higher-order pruning that accounts for expert cooperation in MoEs

HOPE: higher-order pruning that accounts for expert cooperation in MoEs

A Hugging Face blog post presents HOPE, a pruning method for mixture-of-experts models that scores experts in pairs rather than scoring each independently. It notes that Qwen3.5-122B has 256 experts per layer, and that expert MLPs alone account for 95% of its ~250 GB footprint while only 8 experts per layer are used per token. The method needs no training and composes with quantization.

Hugging Face Blog13 d ago
Read original ↗