arXiv:2605.22015cs.CVcs.AR2026-05

用输出引导匹配提升视频生成加速,实现4.5倍速度提升

ORBIS: Output-Guided Token Reduction with Distribution-Aware Matching for Video Diffusion Acceleration

论文配图:ORBIS: Output-Guided Token Reduction with Distribution-Aware Matching for Video Diffusion Acceleration
图 1 · 摘自论文原文
  • 利用前一时刻输出激活计算更准的跨标记相似性
  • 提出分布感知匹配算法,实现2倍于顶尖方法的标记压缩率
  • 软硬件协同设计,仅占2.4%面积却带来79.3%能效降低

扩散变换器(DiT)已成为生成高质量图像与视频的强大架构。在视频生成中,三维时空注意力使标记长度随帧数线性增长,显著增加计算开销。现有标记压缩方法依赖不准确的相似性估计和轻量匹配算法,导致匹配质量差且加速有限。为此,我们提出ORBIS,一种面向视频DiT的软硬件协同加速器。ORBIS利用前一时刻的输出激活获得更精确的标记间相似性,大幅提高匹配质量并支持更高压缩率。进一步提出分布感知标记匹配(DATM)算法,捕捉全局标记分布并显式最小化标记对损失以获取额外收益。为完全隐藏DATM延迟,设计专用深度流水线硬件,并通过量化将硬件开销降至仅2.4%,精度损失可忽略。大量实验表明,ORBIS相比最先进方法AsymRnR实现约2倍更高的标记压缩率,相较NVIDIA A100 GPU最高提升4.5倍速度,能耗降低79.3%。

原文摘要 · Abstract (English)

Diffusion Transformer (DiT) has emerged as a powerful model architecture for generating high-quality images and videos. In the case of video DiT, 3D Spatio-Temporal Attention increases token length in proportion to the number of frames, sharply increasing computational cost. Token reduction methods mitigate this cost by exploiting spatial redundancy, but existing approaches rely on inaccurate similarity estimates and lightweight matching algorithms, resulting in poor matching quality and only marginal acceleration. To overcome these limitations, we propose ORBIS, an SW-HW co-designed accelerator for video DiT. ORBIS leverages the output activation from the previous timestep to obtain more accurate inter-token similarity, substantially improving matching quality and enabling a higher token reduction ratio. We further introduce a Distribution-Aware Token Matching (DATM) algorithm that captures global token distribution and explicitly minimizes token-pair loss for additional gains. To fully hide DATM latency, we design specialized, deeply pipelined hardware and minimize its hardware cost through quantization, occupying only 2.4% of total area with negligible accuracy loss. Extensive experiments show that ORBIS achieves about 2x higher token reduction ratio than the state-of-the-art approach, AsymRnR, while delivering up to 4.5x speedup and 79.3% energy reduction compared to an NVIDIA A100 GPU.

视频生成扩散模型加速器

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。