用混合注意力提升视频生成效率，让长视频生成更快更省资源。
ReHyAt: Recurrent Hybrid Attention for Video Diffusion Transformers
- 结合softmax与线性注意力，实现分块递归计算，内存恒定。
- 训练成本降为160小时（原模型的1%），质量仍达顶尖水平。
- 适合想高效训练或部署长视频生成模型的研究者和开发者。
近期视频扩散模型转向基于Transformer的架构，虽达到顶尖生成质量，但面临二次方复杂度的注意力瓶颈，严重限制长序列扩展。本文提出ReHyAt，一种循环混合注意力机制，融合softmax注意力的保真度与线性注意力的高效性，支持分块递归重构，实现恒定内存占用。相比同期纯线性模型SANA Video，ReHyAt的混合设计可从现有softmax模型高效蒸馏，训练成本降低两个数量级至约160 GPU小时，同时保持竞争力。其轻量蒸馏与微调流程可推广至未来双向softmax模型。在VBench和VBench-2.0上的实验及人工偏好测试表明，ReHyAt在将注意力开销从二次方降至线性的同时，实现顶尖视频质量，推动长时视频与设备端生成的实际可扩展性。项目主页见https://qualcomm-ai-research.github.io/rehyat。
原文摘要 · Abstract (English)
Recent advances in video diffusion models have shifted towards transformer-based architectures, achieving state-of-the-art video generation but at the cost of quadratic attention complexity, which severely limits scalability for longer sequences. We introduce ReHyAt, a Recurrent Hybrid Attention mechanism that combines the fidelity of softmax attention with the efficiency of linear attention, enabling chunk-wise recurrent reformulation and constant memory usage. Unlike the concurrent linear-only SANA Video, ReHyAt's hybrid design allows efficient distillation from existing softmax-based models, reducing the training cost by two orders of magnitude to ~160 GPU hours, while being competitive in the quality. Our light-weight distillation and finetuning pipeline provides a recipe that can be applied to future state-of-the-art bidirectional softmax-based models. Experiments on VBench and VBench-2.0, as well as a human preference study, demonstrate that ReHyAt achieves state-of-the-art video quality while reducing attention cost from quadratic to linear, unlocking practical scalability for long-duration and on-device video generation. Project page is available at https://qualcomm-ai-research.github.io/rehyat.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。