提出高效经济的层剪枝框架，显著降低大模型推理开销且保持高精度。
E$^3$-Pruner: Towards Efficient, Economical, and Effective Layer Pruning for Large Language Models
- 用可微掩码优化与门控顶K采样，精准搜索剪枝结构。
- 在Qwen3-32B上剪掉25%层，精度仅降0.8%，达96%。
- 训练仅需0.5B token，推理加速1.33倍，适合部署优化。
随着大语言模型规模持续增长，层剪枝因其硬件友好性成为模型压缩的重要方法。然而现有方法难以同时应对性能下降、训练成本高和加速有限等实际挑战。为此，我们提出E$^3$-Pruner，一个任务有效、训练经济、推理高效的层剪枝框架。该框架引入两项核心创新：(1) 基于Gumbel-TopK采样的可微掩码优化方法，实现高效精确的剪枝掩码搜索；(2) 基于熵感知的自适应知识蒸馏策略，提升任务性能。在多种模型架构与基准上的大量实验表明，该方法优于当前最先进方法。值得注意的是，在对Qwen3-32B剪枝25%层时，其在MATH-500数据集上达到96%准确率，仅比原模型（96.8%）低0.8%，优于现有SOTA（95%），同时仅消耗0.5B tokens（为后训练数据量的0.5%），实现1.33×推理加速。
原文摘要 · Abstract (English)
With the increasing size of large language models, layer pruning has gained increased attention as a hardware-friendly approach for model compression. However, existing layer pruning methods struggle to simultaneously address key practical deployment challenges, including performance degradation, high training costs, and limited acceleration. To overcome these limitations, we propose \name, a task-\underline{E}ffective, training-\underline{E}conomical and inference-\underline{E}fficient layer pruning framework. \namespace introduces two key innovations: (1) a differentiable mask optimization method using a Gumbel-TopK sampler, enabling efficient and precise pruning mask search; and (2) an entropy-aware adaptive knowledge distillation strategy that enhances task performance. Extensive experiments over diverse model architectures and benchmarks demonstrate the superiority of our method over state-of-the-art approaches. Notably, \namespace achieves 96\% accuracy, a mere 0.8\% drop from the original model (96.8\%) on MATH-500 when pruning 25\% layers of Qwen3-32B, outperforming existing SOTA (95\%), with a 1.33$\times$ inference speedup by consuming merely 0.5B tokens (0.5\% of the post-training data volume).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。