用可微SVD压缩大模型，突破传统量化剪枝局限
Dobi-SVD: Differentiable SVD for LLM Compression and Some New Perspectives
- 将SVD应用于激活值截断，而非仅作为优化距离
- 提出自动确定每层权重最优截断位置的方法
- 解决SVD信息丢失问题，适合追求高效模型部署者
我们提出一种基于SVD的大语言模型压缩新方法，突破了传统量化和剪枝的局限。指出最优SVD应用在于截断激活值，而非仅将其作为优化距离。针对SVD压缩中的三个关键挑战：如何为每个权重矩阵确定最优激活截断位置；如何高效重建截断后的权重矩阵；以及如何缓解因SVD固有的“注入”特性导致的信息损失，我们提出了Dobi-SVD。该方法建立了一种新的、有理论依据的SVD压缩范式。
原文摘要 · Abstract (English)
We provide a new LLM-compression solution via SVD, unlocking new possibilities for LLM compression beyond quantization and pruning. We point out that the optimal use of SVD lies in truncating activations, rather than merely using activations as an optimization distance. Building on this principle, we address three critical challenges in SVD-based LLM compression: including (1) How can we determine the optimal activation truncation position for each weight matrix in LLMs? (2) How can we efficiently reconstruct the weight matrices based on truncated activations? (3) How can we address the inherent "injection" nature that results in the information loss of the SVD? We propose Dobi-SVD, which establishes a new, principled approach to SVD-based LLM compression.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。