提出可扩展的混合精度量化框架，实现低比特下高效部署大模型。
ScaleBITS: Scalable Bitwidth Search for Hardware-Aligned Mixed-Precision LLMs
- 基于敏感度分析与双向通道重排，实现硬件对齐的块级权重划分。
- 在4比特以下极限压缩中，性能比统一精度提升36%，优于现有方法13%。
- 无需运行时开销，适合边缘设备等资源受限场景的模型部署。
后训练权重量化对于降低大语言模型（LLMs）的内存和推理成本至关重要，但将平均精度降至4比特以下仍具挑战性，主要因权重敏感性高度非均匀，且缺乏系统性的精度分配策略。现有方法采用不规则细粒度混合精度，带来高运行时开销，或依赖启发式、受严格约束的精度分配。本文提出ScaleBITS，一种可在内存预算下自动、细粒度分配比特宽度的混合精度量化框架，同时保持硬件效率。通过新的敏感度分析，引入硬件对齐的块级权重划分机制，结合双向通道重排。将全局比特分配建模为约束优化问题，并设计可扩展的贪心算法近似解法，实现端到端的原理化分配。实验表明，ScaleBITS在超低比特场景下显著优于统一精度量化（最高+36%），并超越现有敏感度感知基线（最高+13%），且无额外运行时开销。
原文摘要 · Abstract (English)
Post-training weight quantization is crucial for reducing the memory and inference cost of large language models (LLMs), yet pushing the average precision below 4 bits remains challenging due to highly non-uniform weight sensitivity and the lack of principled precision allocation. Existing solutions use irregular fine-grained mixed-precision with high runtime overhead or rely on heuristics or highly constrained precision allocation strategies. In this work, we propose ScaleBITS, a mixed-precision quantization framework that enables automated, fine-grained bitwidth allocation under a memory budget while preserving hardware efficiency. Guided by a new sensitivity analysis, we introduce a hardware-aligned, block-wise weight partitioning scheme, powered by bi-directional channel reordering. We formulate global bitwidth allocation as a constrained optimization problem and develop a scalable approximation to the greedy algorithm, enabling end-to-end principled allocation. Experiments show that ScaleBITS significantly improves over uniform-precision quantization (up to +36%) and outperforms state-of-the-art sensitivity-aware baselines (up to +13%) in ultra-low-bit regime, without adding runtime overhead.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。