通过统计校准提升激活稀疏性，实现大模型推理加速
Post-Training Statistical Calibration for Higher Activation Sparsity
- 提出后训练激活剪枝框架，基于输入激活分布进行通用稀疏化
- 使用模式中心化技术，使模型在不降质前提下实现1.5倍推理提速
- 适配多种模型架构，尤其适合部署时追求高效推理的场景
我们提出统计校准激活剪枝（SCAP），一种后训练激活剪枝框架，可泛化地对全连接层的输入激活进行稀疏化，适用于各类Transformer模型。该方法引入简单的模式中心化技术，预先校准激活分布，以最大化后训练阶段的稀疏性。实验表明，相比已有方法，SCAP在保持相同模型质量的前提下，相较CATS实现1.5倍的大型语言模型解码速度提升，展现出稳健的帕累托效率。其有效性在多种模型上得到验证，包括近期Transformer解码器、MoE、Mamba2、编码器Transformer及预量化模型，凸显其实际应用价值与可扩展性。代码已开源：https://github.com/IntelLabs/SCAP。
原文摘要 · Abstract (English)
We present Statistical Calibrated Activation Pruning (SCAP), a post-training activation pruning framework that (1) generalizes sparsification by input activations of Fully-Connected layers for generic and flexible application across Transformers, and (2) features a simple Mode-Centering technique to pre-calibrate activation distributions for maximizing post-training sparsity. Our results demonstrate robust Pareto efficiency compared to prior methods, translating to a 1.5x additional LLM decoding speedup against CATS at iso model quality. SCAP effectiveness is empirically verified across a wide range of models, including recent Transformer Decoders, MoE, Mamba2, Encoding Transformer, and pre-quantized models, highlighting its practicality and scalability. The code is available at: https://github.com/IntelLabs/SCAP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。