用大模型自动找量化代理,无需人工和训练。
Revolutionizing Mixed Precision Quantization: Towards Training-free Automatic Proxy Discovery via Large Language Models
- 用大模型和进化搜索自动发现量化代理
- 在主流基准上达到顶尖性能
- 适合想省去调参的人和研究者
混合精度量化(MPQ)可缓解深度神经网络(DNN)的内存溢出(OOM)问题,受到越来越多关注。传统方法要么依赖昂贵的可微优化搜索,效率低且不灵活;要么由专家手动设计代理(如HAWQ),耗时且需专业知识。本文提出无需人类专家或训练的全新框架TAP(Training-free Automatic Proxy),利用大语言模型(LLM)与进化搜索策略,自动发现适配MPQ的优质代理。为弥合黑盒LLM与复杂MPQ任务间的差距,引入轻量级基于直接偏好优化(DPO)的策略控制器,根据适应度信号动态调整三种提示模板的选择概率,无需微调LLM。该机制形成任务感知反馈回路,持续提升代理生成质量。在主流基准上的大量实验表明,TAP性能达当前最优。我们相信,TAP将为MPQ社区提供大模型驱动算法的新视角。
原文摘要 · Abstract (English)
Mixed-Precision Quantization (MPQ) liberates Deep Neural Networks (DNNs) from the Out-Of-Memory (OOM) bottleneck and has garnered increasing research attention. However, conventional methods either rely on costly differentiable optimization search, which is neither efficient nor flexible, or learn a quantized DNN from a proxy (e.g., HAWQ) manually designed by human experts, which is labor-intensive and requires extensive expert knowledge. Can we design a proxy without involving any human experts or training? In this paper, we provide an affirmative answer by proposing a novel Large Language Model (LLM)-driven Training-free Automatic Proxy (dubbed TAP) discovery framework. It reforms the design paradigm of MPQ by utilizing LLMs and evolutionary search strategies to automatically find superior TAP tailored for MPQ. In addition, to bridge the gap between black-box LLMs and the challenging MPQ task, we introduce a lightweight Direct Preference Optimization (DPO)-based strategy controller that dynamically reweights the selection probabilities of the three prompt templates for evolutionary search strategies according to fitness signals, without fine-tuning the LLM. This forms a task-aware feedback loop that improves proxy generation across evolutions. Extensive experiments on mainstream benchmarks demonstrate that TAP achieves state-of-the-art performance. Finally, we believe that our TAP will significantly contribute to the MPQ community by providing a new perspective on LLM-driven design algorithms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。