arXiv:2606.06309cs.CV2026-06

不训练即可加速视频生成,关键帧精修,其他帧跳步省算力

RhymeFlow: Training-Free Acceleration for Video Generation with Asynchronous Denoising Flow Scheduling

论文配图:RhymeFlow: Training-Free Acceleration for Video Generation with Asynchronous Denoising Flow Scheduling
图 1 · 摘自论文原文
  • 只对关键帧做完整去噪,非关键帧跳过部分步骤
  • 在10秒视频上提速2.3倍,图像质量下降<2%
  • 无需训练,适配现有扩散模型,适合实时视频生成

基于扩散Transformer(DiT)的视频生成模型在视频合成上表现卓越,但因3D注意力的二次复杂度导致推理延迟高、计算开销大。现有加速方法主要通过稀疏注意力和键值缓存降低单步计算量,但仍严格遵循标准扩散流程:目标视频序列中每帧都需经历完整的密集去噪过程。我们观察到,相邻帧在内容与运动上具有对应性,当关键帧锚定语义转折点后,其余帧的中间状态往往呈现可预测轨迹,表明对自然视频数据而言,这种统一的密集去噪本质冗余。为此,我们提出 extbf{RhymeFlow}——一种无需训练的框架,将不同帧的去噪轨迹解耦:首先识别主导潜在语义演化的稀疏关键帧;仅对关键帧进行完整逐步去噪以保证结构完整性,非关键帧则逐步跳过去噪步骤以降低计算成本。由于非关键帧跳步会破坏关键帧去噪中的时序一致性,导致视觉退化,我们进一步引入潜空间轨迹投影模块,使关键帧能与完整且时序一致的序列表征交互。在当前主流DiT-based视频生成模型上的大量实验表明,该方法在保持更高视觉质量的同时,显著优于现有基线,推理速度更快。

原文摘要 · Abstract (English)

Video generation models based on Diffusion Transformers (DiTs) have achieved remarkable performance in video synthesis, yet they suffer from high inference latency and computational costs due to the quadratic complexity of 3D attention. Existing acceleration methods primarily reduce computational complexity within each individual denoising steps through techniques such as sparse attention and KV-caching. However, they rigidly adhere to the inherent constraint of the standard diffusion pipeline: every frame in the target video sequence must be subjected to a complete, dense denoising process across all diffusion timesteps. We observe that due to the corresponding contents and motions among adjacent frames, when keyframes with critical semantic transitions are anchored, the intermediate states of others often follow more predictable trajectories, which indicates that such uniform, dense denoising process is inherently redundant for natural video data. To this end, we introduce \textbf{RhymeFlow}, a training-free framework that decouples the denoising trajectories of different frames. Specifically, we first identify a sparse set of pivotal key frames that dominate the latent semantic evolution. Then, only these keyframes undergo dense, step-by-step denoising to ensure structural integrity, while non-keyframes progressively skip denoising steps to minimize computational cost. Since skipped intermediate states of non-keyframes break the temporal coherence in keyframe denoising steps, leading to visual degradation, we further introduce a latent trajectory projection module, which enables keyframes to interact with a complete and temporally consistent sequence representation. Extensive experiments on current DiT-based video generation models demonstrate our method outperforms existing baselines with higher inference speed and better visual quality.

视频生成扩散模型加速推理无训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。