提出新方法实现2比特量化，显著降低大模型内存占用。
Exploring Model Invariance with Discrete Search for Ultra-Low-Bit Quantization
- 通过离散搜索同时探索多种模型不变性，挖掘协同效应。
- 在2比特量化下优于现有先进方法，兼容主流量化技术。
- 适合关注模型压缩与低比特部署的研究者和工程师。
大型语言模型因广泛应用而持续增大，亟需降低内存占用以提升可访问性。后训练量化是一种无需重训练即可用较少比特（如4-8比特）表示模型的常用技术。然而，在超低比特设置（如2比特）下进行量化仍具挑战性。本文提出InvarExplore框架，系统性地同时探索多种模型不变性，充分利用各类不变性间的协同效应。重要的是，InvarExplore采用离散搜索算法，首次实现了对排列不变性的有效探索，因其无法通过梯度方法优化而长期被忽视。实验表明，InvarExplore可兼容现有最先进方法，在强基线基础上实现额外性能提升。
原文摘要 · Abstract (English)
Large language models have been increasing in size due to their success in a wide range of applications. This calls for a pressing need to reduce memory usage to make them more accessible. Post-training quantization is a popular technique which uses fewer bits (e.g., 4--8 bits) to represent the model without retraining it. However, it remains a challenging task to perform quantization in an ultra-low-bit setup (e.g., 2 bits). In this paper, we propose InvarExplore, a unified framework that systematically explores different model invariance at the same time, allowing us to take advantage of the synergy between each type of invariance. Importantly, InvarExplore features a discrete search algorithm that enables us to explore permutation invariance, which is under-studied as it cannot be optimized with gradient-based methods. Results show that InvarExplore is compatible with existing state-of-the-art methods, achieving an add-on performance improvement over strong competing methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。