用余弦兰佐斯双对角化压缩大模型权重，降低内存占用。
Low-Rank Tensor Approximation of Weights in Large Language Models via Cosine Lanczos Bidiagonalization
- 基于cproduct的张量结构，在变换域中联合近似权重切片。
- 相比传统SVD，能更高效捕捉多维相关性，实现低秩压缩。
- 适合需要部署大模型但资源受限的场景。
大型语言模型（LLMs）在多种自然语言任务中表现出色，但面临极高的内存占用和计算成本。本文提出一种基于cproduct的张量压缩框架，用于计算权重张量的低秩近似。首先，利用cproduct的代数结构，将嵌入层、注意力投影和前馈网络中的权重张量表示在变换域中，使各前向切片可被低秩张量因子联合近似。该方法实现了计算高效的压缩，能够超越传统SVD方法，挖掘多维相关性，从而有效降低模型存储与计算开销。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have demonstrated remarkable capabilities across diverse natural language tasks but suffer from extremely large memory footprints and computational costs. In this paper, we introduce a tensor compression framework based on the cproduct for computing low rank approximation In the first part of our approach, we leverage the algebraic structure of the cproduct to represent weight tensors such as those in embedding layers, attention projections, and feed forward networks in a transform domain where frontal slices can be jointly approximated by low rank tensor factors. This enables computationally efficient compression that exploits multidimensional correlations beyond traditional SVD methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。