arXiv:2512.23258cs.CV2025-12中稿 · ed被引 2

提出新方法提升扩散模型加速时的生成质量,解决缓存误差问题。

Plug-and-Play Fidelity Optimization for Diffusion Transformer Acceleration via Cumulative Error Minimization

  • 通过累积误差最小化设计动态缓存策略,自适应调整加速过程
  • 在多个模型上实现显著质量提升,部分超越原始未加速性能
  • 无需额外计算开销,可适配任意加速预算和量化模型

尽管扩散变换器(DiT)已成为图像与视频生成的主流架构,其迭代去噪过程导致推理速度慢,限制了广泛应用。基于缓存的方法可实现无训练加速,但存在显著计算误差。现有方法虽采用剪枝或预测等误差修正策略,却因固定缓存机制无法适应去噪过程中复杂的误差变化,制约了纠错潜力。为此,本文提出一种新型保真度优化插件CEM,通过累积误差最小化实现对现有纠错方法的改进。CEM预先定义误差以刻画模型对加速的敏感性,受时间步与缓存间隔共同影响。基于此先验,构建含累积误差近似的动态规划算法,实现缓存误差最小化,显著提升生成保真度。CEM具有模型无关性与强泛化能力,适用于任意加速预算。其可无缝集成至现有纠错框架与量化模型中,不引入额外计算开销。在九个生成模型及量化方法、三个任务上的大量实验表明,CEM显著提升现有加速模型的生成质量,并在FLUX.1-dev、PixArt-α、StableDiffusion1.5和Hunyuan上超越原始生成性能。代码已公开于https://github.com/leaves162/CEM。

原文摘要 · Abstract (English)

Although Diffusion Transformer (DiT) has emerged as a predominant architecture for image and video generation, its iterative denoising process results in slow inference, which hinders broader applicability and development. Caching-based methods achieve training-free acceleration, while suffering from considerable computational error. Existing methods typically incorporate error correction strategies such as pruning or prediction to mitigate it. However, their fixed caching strategy fails to adapt to the complex error variations during denoising, which limits the full potential of error correction. To tackle this challenge, we propose a novel fidelity-optimization plugin for existing error correction methods via cumulative error minimization, named CEM. CEM predefines the error to characterize the sensitivity of model to acceleration jointly influenced by timesteps and cache intervals. Guided by this prior, we formulate a dynamic programming algorithm with cumulative error approximation for strategy optimization, which achieves the caching error minimization, resulting in a substantial improvement in generation fidelity. CEM is model-agnostic and exhibits strong generalization, which is adaptable to arbitrary acceleration budgets. It can be seamlessly integrated into existing error correction frameworks and quantized models without introducing any additional computational overhead. Extensive experiments conducted on nine generation models and quantized methods across three tasks demonstrate that CEM significantly improves generation fidelity of existing acceleration models, and outperforms the original generation performance on FLUX.1-dev, PixArt-$α$, StableDiffusion1.5 and Hunyuan. Our code is released publicly at https://github.com/leaves162/CEM.

扩散模型加速推理保真度优化动态缓存

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。