通过三步压缩法，在不损失性能前提下让大模型内存减少75%。
SPQ: An Ensemble Technique for Large Language Model Compression
- 融合奇异值分解、激活剪枝和8位量化，分层互补压缩模型
- 在LLaMA-2-7B上实现75%内存压缩，困惑度从5.47降至4.91
- 比GPTQ更快更省内存，适合资源受限环境部署
本文提出一种名为SPQ（SVD-Pruning-Quantization）的集成压缩技术，用于大语言模型（LLM）压缩。该方法结合保留方差的奇异值分解（SVD）、基于激活的剪枝和训练后线性量化，分别针对MLP层冗余神经元、注意力投影高维性及所有线性层的存储开销。在相同压缩率下，SPQ在困惑度上优于单一方法（仅SVD、仅剪枝或仅量化）。应用于LLaMA-2-7B时，可实现最高75%内存压缩，同时保持甚至提升性能：WikiText-2困惑度由5.47降至4.91，并在C4、TruthfulQA和GSM8K等下游任务中保持准确率。相比强基线GPTQ和SparseGPT，SPQ在6.86 GB内存下表现相当，但内存占用更低（7.16 GB），推理吞吐量提升最多达1.9倍，显著增强实际部署可行性。该方法通过分层且互补的压缩策略，为内存受限场景下的大模型落地提供有效方案。代码已开源。
原文摘要 · Abstract (English)
This study presents an ensemble technique, SPQ (SVD-Pruning-Quantization), for large language model (LLM) compression that combines variance-retained singular value decomposition (SVD), activation-based pruning, and post-training linear quantization. Each component targets a different source of inefficiency: i) pruning removes redundant neurons in MLP layers, ii) SVD reduces attention projections into compact low-rank factors, iii) and 8-bit quantization uniformly compresses all linear layers. At matched compression ratios, SPQ outperforms individual methods (SVD-only, pruning-only, or quantization-only) in perplexity, demonstrating the benefit of combining complementary techniques. Applied to LLaMA-2-7B, SPQ achieves up to 75% memory reduction while maintaining or improving perplexity (e.g., WikiText-2 5.47 to 4.91) and preserving accuracy on downstream benchmarks such as C4, TruthfulQA, and GSM8K. Compared to strong baselines like GPTQ and SparseGPT, SPQ offers competitive perplexity and accuracy while using less memory (6.86 GB vs. 7.16 GB for GPTQ). Moreover, SPQ improves inference throughput over GPTQ, achieving up to a 1.9x speedup, which further enhances its practicality for real-world deployment. The effectiveness of SPQ's robust compression through layer-aware and complementary compression techniques may provide practical deployment of LLMs in memory-constrained environments. Code is available at: https://github.com/JiaminYao/SPQ_LLM_Compression/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。