用大模型自动生成并优化扩散模型加速代码,提升推理效率。
DiffBench Meets DiffAgent: End-to-End LLM-Driven Diffusion Acceleration Code Generation
- 大模型驱动的闭环框架,自动设计加速策略与代码。
- 在多种架构上实测,加速效果显著优于现有方法。
- 适合需要高效部署扩散模型的研究者与开发者。
扩散模型在图像与视频生成中取得巨大成功,但其多步推理过程带来巨大计算开销,制约实际部署。加速扩散模型至关重要,但如何组合多种加速技术仍是难题。为此,我们提出一个由大语言模型驱动的自动化加速代码生成与评估框架。首先,构建DiffBench,一个涵盖多种扩散架构、优化组合与部署场景的三阶段自动化评估基准。其次,提出DiffAgent,一个能为任意扩散模型生成最优加速策略与代码的智能体。DiffAgent采用闭环工作流,通过规划、调试组件与代码生成组件协同迭代,并利用遗传算法从执行环境提取性能反馈以指导后续优化。大量实验表明,DiffBench可全面评估生成代码,而DiffAgent在生成有效加速策略方面显著优于现有LLM。
原文摘要 · Abstract (English)
Diffusion models have achieved remarkable success in image and video generation. However, their inherently multiple step inference process imposes substantial computational overhead, hindering real-world deployment. Accelerating diffusion models is therefore essential, yet determining how to combine multiple model acceleration techniques remains a significant challenge. To address this issue, we introduce a framework driven by large language models (LLMs) for automated acceleration code generation and evaluation. First, we present DiffBench, a comprehensive benchmark that implements a three stage automated evaluation pipeline across diverse diffusion architectures, optimization combinations and deployment scenarios. Second, we propose DiffAgent, an agent that generates optimal acceleration strategies and codes for arbitrary diffusion models. DiffAgent employs a closed-loop workflow in which a planning component and a debugging component iteratively refine the output of a code generation component, while a genetic algorithm extracts performance feedback from the execution environment to guide subsequent code refinements. We provide a detailed explanation of the DiffBench construction and the design principles underlying DiffAgent. Extensive experiments show that DiffBench offers a thorough evaluation of generated codes and that DiffAgent significantly outperforms existing LLMs in producing effective diffusion acceleration strategies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。