arXiv:2508.16211cs.CV2025-08AAAI被引 20

用微分方程思路优化扩散模型推理,大幅提速且保持画质

Forecast then Calibrate: Feature Caching as ODE for Efficient Diffusion Transformers

  • 将隐藏特征序列建模为微分方程,实现更稳定的跨步预测
  • 无额外训练下达到5.5倍加速,视频生成最高达6.45倍
  • 适合需要高效推理的图像/视频生成场景

扩散Transformer(DiTs)在高保真图像与视频生成中表现卓越。为降低其高昂计算成本,特征缓存技术通过复用前时间步的隐藏表示来加速推理。然而现有方法在高加速比下常因长步预测固有不稳定性导致生成质量下降。本文从常微分方程(ODE)视角建模隐藏特征序列,将层表示沿轨迹视为特征-ODE。我们指出现有缓存策略在大跳步间隔下难以鲁棒融合历史特征,是质量退化的原因。为此提出FoCa(Forecast-then-Calibrate),将特征缓存视为求解特征-ODE问题。在图像合成、视频生成和超分辨率任务上的大量实验表明,该方法在激进加速下依然有效:无需额外训练,对FLUX实现5.50倍近无损加速,对HunyuanVideo达6.45倍,对Inf-DiT达3.17倍,对DiT在4.53倍加速下仍保持高质量。

原文摘要 · Abstract (English)

Diffusion Transformers (DiTs) have demonstrated exceptional performance in high-fidelity image and video generation. To reduce their substantial computational costs, feature caching techniques have been proposed to accelerate inference by reusing hidden representations from previous timesteps. However, current methods often struggle to maintain generation quality at high acceleration ratios, where prediction errors increase sharply due to the inherent instability of long-step forecasting. In this work, we adopt an ordinary differential equation (ODE) perspective on the hidden-feature sequence, modeling layer representations along the trajectory as a feature-ODE. We attribute the degradation of existing caching strategies to their inability to robustly integrate historical features under large skipping intervals. To address this, we propose FoCa (Forecast-then-Calibrate), which treats feature caching as a feature-ODE solving problem. Extensive experiments on image synthesis, video generation, and super-resolution tasks demonstrate the effectiveness of FoCa, especially under aggressive acceleration. Without additional training, FoCa achieves near-lossless speedups of 5.50 times on FLUX, 6.45 times on HunyuanVideo, 3.17 times on Inf-DiT, and maintains high quality with a 4.53 times speedup on DiT.

扩散模型推理加速特征缓存微分方程

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。