联合优化模型结构与量化，让大模型在边缘设备上更快更准
LLM Compression with Jointly Optimizing Architectural and Quantization choices
- 用可微分搜索同时选模型结构和混合精度量化方案
- 推理速度比传统方法快1.4倍，或在同等延迟下准确率高6%
- 适合想在移动端部署大模型的研究者和工程师
部署大语言模型（LLMs）面临显著的内存和计算挑战。虽然一些方法通过从头训练小型模型来缓解问题，但这类方法需要大量GPU训练。对预训练的LLM进行压缩以适配边缘设备是一种更具吸引力的替代方案。除了剪枝和量化外，神经架构搜索（NAS）能有效实现压缩，但以往的NAS方法常受限于搜索空间，并将架构设计与量化分离。本文提出一种可微分的NAS框架，全面探索搜索空间，并联合优化线性层的模型架构与混合精度量化策略。实验表明，该方法在准确率-延迟权衡上表现优异：在相近准确率下，推理速度最高提升1.4倍；或在相同延迟下，七项推理任务平均准确率提高6%。
原文摘要 · Abstract (English)
Deploying large language models (LLMs) is challenging due to their significant memory and computational requirements. While some methods address this by developing small or tiny language models from scratch, these approaches demand extensive GPU training. Compressing pre-trained LLMs for edge devices offers a compelling alternative. Beyond pruning and quantization, Neural Architecture Search (NAS) enables effective compression, yet prior NAS approaches often limit the search space and decouple architecture from quantization. We introduce a differentiable NAS framework that explores the entire space and jointly optimizes architectural configurations alongside mixed-precision quantization for linear layers of LLMs. Experiments demonstrate superior accuracy-latency trade-offs: our models achieve up to 1.4x faster inference than sequential NAS-then-quantization baselines at comparable accuracy, or up to 6% higher average accuracy across seven reasoning tasks at equivalent latency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。