通过线性多步预测加速扩散Transformer，保持生成质量的同时显著降低延迟。
Predict to Skip: Linear Multistep Feature Forecasting for Efficient Diffusion Transformers
- 基于历史特征线性外推预测未来输出，避免重复使用导致的误差积累。
- 在图像与视频生成模型上实现最高5.54倍的推理加速，质量损失可忽略。
- 适合需要高效生成且对实时性要求高的场景，如视频创作、交互式设计。
扩散Transformer（DiT）已成为高质量图像与视频生成的主流骨干架构，但其迭代去噪过程带来高昂计算成本。现有无训练加速方法依赖特征缓存与重用，假设特征在时间上稳定。然而，跨多步重用特征可能导致潜在空间漂移和视觉退化。我们观察到模型输出在整个扩散轨迹中变化平滑，因此可进行有原则的预测而非简单重用。基于此，我们提出PrediT——一种无训练加速框架，将特征预测建模为线性多步问题。采用经典线性多步方法，利用历史信息预测未来输出，并引入校正器在高动态区域激活以防止误差累积。动态步长调节机制通过监控特征变化率自适应调整预测范围。大量实验表明，该方法在多种基于DiT的图像与视频生成模型上实现了最高5.54倍的延迟降低，同时质量退化极小。
原文摘要 · Abstract (English)
Diffusion Transformers (DiT) have emerged as a widely adopted backbone for high-fidelity image and video generation, yet their iterative denoising process incurs high computational costs. Existing training-free acceleration methods rely on feature caching and reuse under the assumption of temporal stability. However, reusing features for multiple steps may lead to latent drift and visual degradation. We observe that model outputs evolve smoothly along much of the diffusion trajectory, enabling principled predictions rather than naive reuse. Based on this insight, we propose \textbf{PrediT}, a training-free acceleration framework that formulates feature prediction as a linear multistep problem. We employ classical linear multistep methods to forecast future model outputs from historical information, combined with a corrector that activates in high-dynamics regions to prevent error accumulation. A dynamic step modulation mechanism adaptively adjusts the prediction horizon by monitoring the feature change rate. Together, these components enable substantial acceleration while preserving generation fidelity. Extensive experiments validate that our method achieves up to $5.54\times$ latency reduction across various DiT-based image and video generation models, while incurring negligible quality degradation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。