提出一种兼顾时序连贯性的视频生成剪枝方法，显著提速且不损失画质。
Temporal Aware Pruning for Efficient Diffusion-based Video Generation
- 通过时间平滑对齐相邻帧的注意力重要性，减少帧间抖动
- 在关键层重选令牌，避免语义偏差累积，提升一致性
- 按阶段动态调整剪枝强度，早期激进剪枝，后期保真优化
基于ViT架构的视频扩散模型虽能生成高质量视频，但因需处理长时空序列，计算成本高。现有基于注意力的剪枝方法多逐帧操作，难以保证视频生成中的时序连贯性，导致背景不一致、闪烁和画质下降。为此，本文提出无需训练的时序感知剪枝方法TAPE：(i) 对相邻帧进行时间平滑，对齐令牌重要性并抑制选择抖动；(ii) 在选定层内重选令牌，使剪枝匹配各层不同的语义聚焦，避免特定区域误差累积；(iii) 采用层级式预算调度，在早期噪声步骤激进剪枝，后期放松以保障保真度。实验表明，TAPE在大幅加速的同时保持高视觉质量，优于已有令牌压缩方法。
原文摘要 · Abstract (English)
Video diffusion models have recently enabled high-quality video generation with ViT-based architectures, but remain computationally intensive because generation requires attention computation over long spatiotemporal sequences. Token pruning has proven effective for ViTs and VLMs. However, most prior pruning methods are attention-based and operate per frame, failing to ensure the vital temporal coherence across frames in video generation tasks. In practice, naively adopting attention-only pruning causes noticeable degradation due to worsened background consistency, flickering, and reduced image quality. To address this, we propose TAPE, a training-free Temporal Aware Pruning for Efficient diffusion-based video generation. TAPE (i) applies temporal smoothing to align token-importance across adjacent frames and suppress selection jitter; and (ii) performs token reselection in selected layers to align token pruning with layers' diverse semantic focus and avoid error accumulation in specific areas; it also (iii) adopt a timestep-level budget scheduling that prunes aggressively at early noisy steps and relaxes pruning during fidelity-critical refinement. The experimental results show that TAPE delivers significant speedups while preserving high visual fidelity, outperforming prior token reduction approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。