用矩阵分解压缩大模型，速度超快且损失小。
CURing Large Models: Compression via CUR Decomposition
- 选重要行和列重构权重矩阵，保留关键信息。
- Llama3.1-8B压缩至7.32B参数，仅耗时129秒。
- 适合追求快速轻量化大模型的开发者。
大型深度学习模型虽表现优异，但资源消耗大，尤其在内存使用上面临挑战。本文提出CURing，一种基于CUR矩阵分解的新式模型压缩方法，将权重矩阵近似为选定列（C）、行（R）与小型连接矩阵（U）的乘积。通过结合权重幅度与激活值的影响来选取关键部分，识别并保留具有信息量的行与列，实现显著的模型压缩，同时保持性能损失极小。例如，该方法在仅129秒内将Llama3.1-8B模型参数减少至7.32B（降幅9%），速度超过先前压缩方法20倍以上。
原文摘要 · Abstract (English)
Large deep learning models have achieved remarkable success but are resource-intensive, posing challenges such as memory usage. We introduce CURing, a novel model compression method based on CUR matrix decomposition, which approximates weight matrices as the product of selected columns (C) and rows (R), and a small linking matrix (U). We apply this decomposition to weights chosen based on the combined influence of their magnitudes and activations. By identifying and retaining informative rows and columns, CURing significantly reduces model size with minimal performance loss. For example, it reduces Llama3.1-8B's parameters to 7.32B (-9%) in just 129 seconds, over 20 times faster than prior compression methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。