Understand this story
On November 9, 2026, the Hugging Face blog introduced the second-order MoE expert pruning method proposed in the HOPE (Higher-Order Pruning of Experts) paper. This method addresses the problem that existing methods independently assign scalar importance scores to each expert and ignore stable co-activation collaboration groups among experts, by selecting the pruning set through a solvable upper-bound quadratic program that minimizes pruning error. The blog states that at a 50% pruning rate, HOPE ranks on average better than all compared methods, and on agentic coding tasks achieves up to +6.1% improvement over the next-best method. Taking Qwen3.5-122B as an example, its 48 MoE layers each have 256 experts, and about 95% of the roughly 250 GB of parameter memory is occupied by expert MLPs, while each token uses only 8 experts per layer. Currently, this work is publicly available only as a blog post, with no subsequent progress yet.
AI-generated from public reports
Latest development · Report
HOPE: A Second-Order MoE Pruning Method That Accounts for Expert CollaborationDevelopments
1 confirmed developments
The paper proposes a second-order expert pruning method, HOPE
Source:Hugging Face
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Nov 9
Hugging FaceFirst-party
HOPE: A Second-Order MoE Pruning Method That Accounts for Expert Collaboration
The Hugging Face blog introduces HOPE (Higher-Order Pruning of Experts), a second-order MoE pruning method that accounts for pairwise expert interactions; at a 50% pruning rate it achieves a better average rank than all compared methods, and on agentic coding tasks it improves by up to +6.1% over the next-best method. Existing methods independently assign each expert a scalar importance score, ignoring the fact that experts form stable co-activation collaboration groups; HOPE selects the pruning set by minimizing a solvable upper bound of the pruning error via quadratic programming. Taking Qwen3.5-122B as an example, its 48 MoE layers each have 256 experts, and of roughly 250 GB of parameter memory, 95% is occupied by expert MLPs, while each token uses only 8 experts per layer.