压缩后无需微调,用线性重建恢复模型精度
GRAIL: Post-hoc Compensation by Linear Reconstruction for Compressed Networks
- 通过格拉姆矩阵与岭回归,压缩后重建隐藏层特征
- 在ResNet/ViT/LLM上提升准确率或困惑度,优于现有方法
- 无需标签和反向传播,适配各类剪枝策略
结构化深度模型压缩方法硬件友好,显著降低内存和推理开销。但在激进压缩下,精度下降常需压缩后微调,而受限于无标签数据或训练成本高,该过程常不切实际。本文提出一种后处理块级补偿方法GRAIL,即在模型压缩后添加一个零微调步骤,利用少量校准数据恢复每个模块的输入输出行为。该方法通过格拉姆矩阵总结隐藏激活,并采用岭回归从压缩后的表示中线性重建原始隐藏表征。重建映射被融合至下游投影权重,上游层保持压缩。该方法对选择器无依赖(支持大小、Wanda、基于格拉姆的选择或折叠),具备数据感知性(仅需几次前向传播,无需梯度或标签),且当格拉姆矩阵接近单位阵时,退化为经典剪枝或折叠(表明通道间相关性弱)。在多种ResNet、ViT及仅解码器的LLM上,GRAIL在实际压缩场景中持续优于无数据和有数据的剪枝/折叠基线,计算开销可控,无需反向传播。代码已开源:https://github.com/TWWinde/GRAIL_Compensation。
原文摘要 · Abstract (English)
Structured deep model compression methods are hardware-friendly and substantially reduce memory and inference costs. However, under aggressive compression, the resulting accuracy degradation often necessitates post-compression finetuning, which can be impractical due to missing labeled data or high training cost. We propose post-hoc blockwise compensation, called GRAIL, a simple zero-finetuning step applied after model compression that restores each block's input-output behavior using a small calibration set. The method summarizes hidden activations via a Gram matrix and applies ridge regression to linearly reconstruct the original hidden representation from the reduced one. The resulting reconstruction map is absorbed into the downstream projection weights, while the upstream layer is compressed. The approach is selector-agnostic (Magnitude, Wanda, Gram-based selection, or folding), data-aware (requiring only a few forward passes without gradients or labels), and recovers classic pruning or folding when the Gram matrix is near identity, indicating weak inter-channel correlations. Across ResNets, ViTs, and decoder-only LLMs, GRAIL consistently improves accuracy or perplexity over data-free and data-aware pruning or folding baselines in practical compression regimes, with manageable overhead and no backpropagation. The code is available at https://github.com/TWWinde/GRAIL_Compensation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。