优化奇异值截断，让大模型压缩更高效。
SVD-LLM V2: Optimizing Singular Value Truncation for Large Language Model Compression
- 按层分配不同压缩比，适应权重冗余差异
- 通过损失优化截断，降低实际压缩损失
- 适用于各类大模型，提升压缩后性能
尽管取得显著进展，大语言模型（LLMs）的实用部署常受限于其庞大体积，亟需有效的压缩技术。奇异值分解（SVD）是一种有前景的压缩方法，但现有基于SVD的压缩方案在减少截断损失方面表现不足，导致压缩模型性能不理想。本文提出SVD-LLM V2，一种基于SVD的大模型压缩方法，通过两种技术优化奇异值截断：首先，利用权重重构的理论截断损失，为不同层的权重矩阵分配独立压缩比，以适应权重冗余异质性；其次，引入损失优化的权重截断策略，确保截断后的奇异值在实践中带来更低且更稳定的截断损失。我们在十个数据集和五种不同规模的LLMs上评估了SVD-LLM V2，结果表明其性能优于现有的最先进SVD压缩方法。代码已开源：https://github.com/AIoT-MLSys-Lab/SVD-LLM
原文摘要 · Abstract (English)
Despite significant advancements, the practical deployment of Large Language Models (LLMs) is often hampered by their immense sizes, highlighting the need for effective compression techniques. Singular Value Decomposition (SVD) is a promising LLM compression technique. However, existing SVD-based compression methods fall short in reducing truncation losses, leading to less competitive performance in compressed models. In this work, we introduce SVD-LLM V2, a SVD-based LLM compression method that optimizes singular value truncation in SVD compression with two techniques. First, SVD-LLM V2 proposes to use theoretical truncation loss of weight matrices to assign a unique compression ratio to each weight matrix at different layers to accommodate weight redundancy heterogeneity. Second, SVD-LLM V2 proposes loss-optimized weight truncation to ensure that the truncated singular values result in a lower and more stable truncation loss in practice. We evaluate SVD-LLM V2 on ten datasets and five LLMs at various scales. Our results show SVD-LLM V2 outperforms state-of-the-art SVD-based LLM compression methods. Our code is available at https://github.com/AIoT-MLSys-Lab/SVD-LLM
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。