arXiv:2603.20492cs.LGcs.CL2026-03

自动组合高效技术,让大模型在不同场景下又快又省。

AE-LLM: Adaptive Efficiency Optimization for Large Language Models

  • 根据任务和硬件自动选最优压缩、推理和微调方案。
  • 平均效率提升2.8倍,准确率仅比基线低1.2%。
  • 适合想快速部署大模型的工程师和研究者。

大语言模型在多领域取得显著成功,但部署面临计算成本高、内存占用大、能耗高等挑战。近期研究表明,没有单一效率技术在所有场景下都最优;例如高效注意力机制、混合专家(MoE)、参数高效微调和量化等方法的效果,因任务特性、资源限制和模型规模而异。基于此,我们提出AE-LLM,一个统一框架,可自动选择并组合最适合特定部署场景的效率技术。该方法构建多目标优化框架,综合考虑准确率、延迟、内存占用和能耗,并纳入硬件约束与任务需求。我们设计了一种高效的搜索算法,在架构、微调和推理三个阶段的效率技术组合空间中探索,识别出帕累托最优配置。在15个模型(0.5B–70B参数)和10个多样化任务上的大量实验表明,相较于静态配置,AE-LLM实现了平均2.8倍的效率提升,同时保持与基线相差不超过1.2%的准确率。此外,该框架在视觉-语言模型上也展现出良好的泛化能力,获得相似的效率增益。本工作为从业者提供了一个自动化工具,用于应对大模型效率优化中的复杂权衡问题。

原文摘要 · Abstract (English)

Large Language Models (LLMs) have achieved remarkable success across diverse applications, yet their deployment remains challenging due to substantial computational costs, memory requirements, and energy consumption. Recent empirical studies have demonstrated that no single efficiency technique is universally optimal; instead, the effectiveness of methods such as efficient attention mechanisms, mixture-of-experts (MoE), parameter-efficient fine-tuning, and quantization varies significantly depending on task characteristics, resource constraints, and model scales. Building upon these insights, we propose AE-LLM, a unified framework that automatically selects and combines optimal efficiency techniques tailored to specific deployment scenarios. Our approach introduces a multi-objective optimization framework that jointly considers accuracy, latency, memory footprint, and energy consumption, while accounting for hardware constraints and task requirements. We develop an efficient search algorithm that explores the combinatorial space of efficiency techniques across architecture, fine-tuning, and inference stages, identifying Pareto-optimal configurations. Extensive experiments across 15 models (0.5B-70B parameters) and 10 diverse tasks demonstrate that AE-LLM achieves an average of $2.8\times$ improvement in efficiency metrics while maintaining competitive accuracy (within 1.2\% of baseline), compared to static efficiency configurations. Furthermore, our framework generalizes effectively to vision-language models, achieving similar efficiency gains. Our contributions provide practitioners with an automated tool for navigating the complex trade-off landscape of LLM efficiency optimization.

大模型优化自动调优效率提升多目标优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。