用简单线性模块修复剪枝后激活值失衡，显著提升剪枝模型性能。
A Simple Linear Patch Revives Layer-Pruned Large Language Models
- 在剪枝层接口引入线性融合操作，同时抑制异常值并对齐激活统计量。
- 在LLaMA-3-8B上剪掉5层后仍保持94.15%原模型性能，优于之前最佳方法4%。
- 仅需5000条无标签数据和单卡30分钟即可进一步优化至95.16%精度。
层剪枝已成为压缩大语言模型的常用方法，但现有方法常导致性能显著下降。我们发现其主要原因是剪枝界面处激活值幅度不匹配：剪枝前后的激活值尺度差异显著，引发分布偏移并传播至后续层。为此，我们提出轻量级、可即插即用的 extsc{LinearPatch} 技术，在剪枝接口通过一次矩阵乘法融合两种操作：(i) Hadamard 变换以抑制特定标记的极端异常值；(ii) 通道级缩放以对齐激活统计特性。在 LLaMA-3-8B 上，剪除 5/32 层后， extsc{LinearPatch} 保留了高达 94.15% 的原始性能，相较此前最优方法提升 4%。通过 5K 无标签样本进行内存高效的离线蒸馏，性能可进一步提升至 95.16%，仅需单卡运行 30 分钟。代码已公开于 https://github.com/chenxinrui-tsinghua/LinearPatch。
原文摘要 · Abstract (English)
Layer pruning has emerged as a widely used technique for compressing large language models (LLMs). However, existing layer pruning approaches often incur substantial performance degradation. We identify the majority of this degradation to a single yet previously overlooked issue: \textit{the mismatch of activation magnitudes at the pruning interface}. The pre-interface activations exhibit significantly different scales from the post-interface ones, causing the distributional shift as it propagates through the remaining layers. To address this issue, we introduce \textsc{LinearPatch}, a lightweight and plug-and-play technique that fuses two operations into one matrix multiply at the pruning interface: (i) a Hadamard transformation that suppresses massive outliers at particular tokens and (ii) a channel-wise scaling that aligns activation statistics. On LLaMA-3-8B, \textsc{LinearPatch} preserves up to \textbf{94.15\%} of the original model's performance when pruning 5 out of 32 layers, outperforming the previous state of the art by \textbf{4\%}. The patch can be further refined with 5K unlabeled samples via memory-efficient offline distillation, pushing the retention to 95.16\% within only 30 minutes on a single GPU. Code is available at https://github.com/chenxinrui-tsinghua/LinearPatch.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。