针对大模型批处理推理,提出新型无训练剪枝方法,提升效率与准确率。
Batch-wise Adaptive Pruning: Periodic Neuron Activation-Aware Weight Pruning for Language Reasoning Model

- 用周期性 top-k 选择替代阈值剪枝,适应批量激活分布变化。
- 在目标稀疏度 50% 下,准确率比当前最优高出 39.7 个百分点。
- 适合需要高吞吐推理的大型语言推理模型部署场景。
大型推理模型(LRMs)通过长链式思维生成实现复杂任务强性能,但推理时计算开销巨大。在生产环境中,批处理推理对高吞吐至关重要,然而现有无训练自适应剪枝方法在此场景下性能严重下降。原因在于:批处理需共享单一剪枝掩码,导致样本间激活聚合,进而使基于阈值的剪枝策略失效——原线下校准的阈值不再匹配聚合后的分布,实际稀疏度漂移,推理任务准确率崩溃。本文提出一种专为批处理推理设计的无训练自适应剪枝方法,包含两部分:第一,以周期性 top-k 选择替代阈值筛选,不受激活分布偏移影响,且每更新周期仅执行一次,保持加速效果;第二,基于重要神经元在长推理中周期性重激活的现象,引入激活记忆机制,跨周期累积重要性,保留重复活跃神经元。在多个推理基准测试中,本方法在批大小为 4、目标稀疏度 50% 的 DeepSeek-R1-Distill-Qwen-7B 上,平均准确率比前序最优方法提升 39.7 个百分点,实际稀疏度达 50% 时相较稠密推理实现 1.40 倍加速。
原文摘要 · Abstract (English)
Large Reasoning Models (LRMs) achieve strong performance on complex tasks through extended chain-of-thought generation, but incur substantial computational costs during inference. In production settings, batched inference is essential for high throughput, yet the existing training-free adaptive pruning methods we evaluate severely degrade in this regime. Because a batch must share a single pruning mask, these methods aggregate activations across samples and then apply threshold-based selection; the threshold, calibrated offline on unaggregated activations, no longer matches the aggregated distribution, so the realized sparsity ratio drifts and accuracy on reasoning tasks collapses under batched inference. In this work, we propose a training-free adaptive pruning method designed specifically for batched inference in LRMs, built on two components. First, we replace threshold-based selection with periodic top-k selection over the aggregated importance scores, which is unaffected by the shift that aggregation induces in the activation distribution, and which runs selection once per update period rather than at every token, preserving the speedup. Second, based on the observation that important neurons re-fire periodically during long reasoning generation, we introduce an activation memory that accumulates importance across update phases so that recurring neurons are retained. Experiments on diverse reasoning benchmarks demonstrate that our method outperforms the previous state-of-the-art adaptive pruning method by 39.7 percentage points in average accuracy at batch size 4 with 50% target sparsity on DeepSeek-R1-Distill-Qwen-7B, and reaches 1.40x speedup over dense inference at 50% actual sparsity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。