自动为大模型分配精度，让模型更小更快还保持高效果。
AMQ: Enabling AutoML for Mixed-precision Weight-Only Quantization of Large Language Models
- 用智能筛选+代理评估，避开海量配置的盲目尝试。
- 在100多层模型上实现低至4位的量化，精度损失小于1%。
- 适合想压缩模型、部署到边缘设备的研究者和工程师。
为实现大语言模型（LLMs）的广泛部署，需在严格内存约束下找到最优性能模型。本文提出AMQ（Automated Mixed-Precision Weight-Only Quantization），一种自动为各层分配不同量化位宽的框架，以在模型质量与内存占用间取得最佳平衡。然而，组合搜索空间超过10^100种可能配置，传统黑盒优化不可行。AMQ通过四大创新克服该挑战：(1) 利用先验知识剪枝无效配置；(2) 引入量化代理，避免搜索过程中的高成本格式转换；(3) 设计质量预测器，大幅降低评估开销；(4) 采用迭代搜索-更新策略，实现快速稳定收敛。集成后，AMQ高效探索质量-效率权衡空间，达到帕累托前沿，生成既紧凑又高性能的LLM。代码已开源：https://github.com/dlwns147/amq。
原文摘要 · Abstract (English)
To enable broader deployment of Large Language Models (LLMs), it is essential to identify the best-performing model under strict memory constraints. We present AMQ, Automated Mixed-Precision Weight-Only Quantization, a framework that assigns layer-wise quantization bit-widths to optimally balance model quality and memory usage. However, the combinatorial search space, with over 10^{100} possible configurations, makes conventional black-box optimization infeasible. AMQ overcomes this challenge through four key innovations:(1) search space pruning using prior knowledge to exclude unpromising configurations, (2) quantization proxy to bypass costly format conversions during search, (3) quality predictor to minimize evaluation overhead, and (4) iterative search-and-update strategy for fast and stable convergence. By integrating these components, AMQ efficiently explores the quality-efficiency landscape, reaching the Pareto frontier and yielding LLMs that are both compact and high-performing. Our code is available at https://github.com/dlwns147/amq.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。