混合压缩不只是叠加：剪枝与量化在 LLM 上的交互效应

What happens when you shrink a language model twice — once by removing some of its weights, once by storing what’s left with fewer bits? You’d guess the damage just adds up. It doesn’t.

Small language models are increasingly used for on-device apps that need to run in tight memory budgets [Zhou et al. 2025]. To shrink them enough, people often combine two tricks: pruning, which removes some of the model’s weights, and quantization, which stores the remaining weights using fewer bits. Recent work has combined both, often finding the mix beats either trick alone [Kang et al. 2025]. But most of these studies only look at the combined result — they don’t ask whether the two tricks are simply adding their effects together, or doing something else entirely.

We wanted to find out.

To check, we borrowed a simple idea from statistics. For each benchmark, we measure four things: the original model, the pruned-only model, the quantized-only model, and the hybrid. Then we compare:

If the two tricks were simply adding up, would land close to zero. If is positive, the hybrid does better than expected. If it’s negative, the hybrid does worse.

We tested this on three small models — Falcon3-1B-Base, LLaMA-3.2-1B, and a quick check on Qwen2.5-1.5B — at different pruning levels and with both 4-bit and 8-bit quantization, all on a single consumer GPU (an RTX 3080 Ti).

The results split sharply by task. On TruthfulQA, the hybrids consistently beat expectations — by about +7 points on Falcon and +12 to +14 points on LLaMA. It’s as if combining the two tricks accidentally made the models more careful about truthfulness.

On reasoning and knowledge tasks, the opposite happened. GSM8K (grade-school math) and MMLU (general knowledge) dropped far more than expected — by 16 to 20 points on Falcon. In several hybrid setups, GSM8K accuracy fell all the way to zero.

We assumed these effects would get worse the more we pruned. They didn’t. Instead, they peaked at moderate pruning (20–30%) and faded back toward normal at 50% pruning — both the TruthfulQA boost and the reasoning drop. More pruning didn’t mean more damage; it meant the surprise effect faded.

We also found the order barely mattered — quantizing before pruning versus after changed results by only a point or two. What mattered was simply whether the two tricks were combined at all.

If your app cares about factual reliability, moderate hybrid compression might actually help. If it needs strong reasoning or broad knowledge, stick with quantization alone — hybrids can quietly wreck more than you’d expect. And if you just want a smaller model, quantization is doing almost all the work: pruning alone barely changes memory use, while 4-bit quantization cuts it by roughly 2 to 3 times.

The bigger lesson: don’t assume compression tricks just add up. Measure them.

We used simple, unstructured pruning without special hardware support, so our memory numbers don’t reflect what’s possible with dedicated sparse hardware. We tested two models closely and one lightly, and none were fine-tuned afterward. Testing more models, hardware-aware pruning, and light fine-tuning (like LoRA) to balance truthfulness gains against reasoning losses are natural next steps.