通过排序发现大模型权重中的隐含低秩结构，实现高效压缩。
EinSort: Sorting is All We Need for Tensorizing LLM
- 基于索引排序自动发现权重中的低秩特性。
- 在权重与KV缓存压缩中重建精度优于现有方法。
- 适合需要轻量化部署大模型的研究者使用。
张量网络为压缩大型神经网络提供了高效表示。通过精心设计形状与拓扑结构，可显著降低内存和计算开销。然而，由于大模型规模巨大且权重分布无序，识别其中隐含的低秩结构仍具挑战。本文提出一种自适应张量化方法，通过索引排序发现目标张量中的内在低秩结构。在权重与KV缓存压缩实验中，该方法相比基线模型实现了更优的重建质量。
原文摘要 · Abstract (English)
Tensor networks provide efficient representations for compressing large neural networks. By carefully designing shapes and topologies, they can significantly reduce memory and computational costs. However, identifying implicit low-rank structures in large foundation models remains challenging due to their enormous scale and un-structured weight distributions. We propose an adaptive tensorization method that discovers inherent low-rank structure in a target tensor by index ordering. Experiments on weight and KV-cache compression demonstrate improved reconstruction quality compared to baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。