用随机种子压缩大模型权重，提速降存不依赖数据。
SeedLM: Compressing LLM Weights into Seeds of Pseudo-Random Generators
- 用线性反馈移位寄存器种子生成权重矩阵，结合压缩系数重构。
- 700亿参数模型4比特压缩后零样本准确率领先现有方法。
- 无需校准数据，适合部署在资源受限设备如FPGA上。
大型语言模型（LLMs）虽推动自然语言处理发展，但因运行成本高难以广泛部署。本文提出一种新型后训练压缩方法SeedLM，利用伪随机生成器的种子编码并压缩模型权重。具体而言，对每一块权重，寻找一个种子输入线性反馈移位寄存器（LFSR），在推理时高效生成随机矩阵，再与压缩系数线性组合以重构权重块。该方法减少内存访问，利用推理期间空闲计算周期，通过牺牲少量计算实现内存密集型任务的加速。与依赖校准数据的先进压缩方法不同，SeedLM为无数据方法，在多种任务间具有良好泛化能力。在特别难压缩的Llama 3 70B模型上，实验显示其4比特和3比特压缩下的零样本准确率显著优于当前最优技术，且性能接近FP16基线。此外，基于FPGA的测试表明，随着模型规模增至700亿参数，4比特SeedLM相较FP16 Llama 2/3基线可达到近4倍加速。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have transformed natural language processing, but face significant challenges in widespread deployment due to their high runtime cost. In this paper, we introduce SeedLM, a novel post-training compression method that uses seeds of pseudo-random generators to encode and compress model weights. Specifically, for each block of weights, we find a seed that is fed into a Linear Feedback Shift Register (LFSR) during inference to efficiently generate a random matrix. This matrix is then linearly combined with compressed coefficients to reconstruct the weight block. SeedLM reduces memory access and leverages idle compute cycles during inference, effectively speeding up memory-bound tasks by trading compute for fewer memory accesses. Unlike state-of-the-art compression methods that rely on calibration data, our approach is data-free and generalizes well across diverse tasks. Our experiments with Llama 3 70B, which is particularly challenging to compress, show that SeedLM achieves significantly better zero-shot accuracy retention at 4- and 3-bit than state-of-the-art techniques, while maintaining performance comparable to FP16 baselines. Additionally, FPGA-based tests demonstrate that 4-bit SeedLM, as model size increases to 70B, approaches a 4x speed-up over an FP16 Llama 2/3 baseline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。