从凸优化出发，实现百亿参数大模型的高效量化压缩。
Foundations of Large Language Model Compression -- Part 1: Weight Quantization
- 基于凸优化理论构建量化框架，确保最优压缩效果。
- 支持百亿级参数模型，可任意指定压缩后模型大小。
- 纯训练后压缩，无需重新训练，适合资源受限部署。
近年来，大语言模型（LLMs）的压缩已成为在资源受限设备上部署语言模型、降低计算成本以及减少大规模AI基础设施环境影响的重要问题。本文从凸优化的角度为大语言模型量化奠定基础，并提出一种基于该理论的量化技术，以实现最佳量化效果。所提出的量化框架CVXQ可扩展至包含数百亿参数的模型，支持用户在无需重新训练的情况下，将模型压缩至任意指定尺寸。CVXQ的参考实现可在github.com/seannz/cvxq获取。
原文摘要 · Abstract (English)
In recent years, compression of large language models (LLMs) has emerged as an important problem to enable language model deployment on resource-constrained devices, reduce computational costs, and mitigate the environmental footprint of large-scale AI infrastructure. In this paper, we lay down the foundation for LLM quantization from a convex optimization perspective and propose a quantization technique that builds on this foundation for optimum quantization outcomes. Our quantization framework, CVXQ, scales to models containing hundreds of billions of weight parameters and provides users with the flexibility to compress models to any specified model size, post-training. A reference implementation of CVXQ can be obtained from github.com/seannz/cvxq.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。