高效生成高质量视频,突破注意力机制计算瓶颈。
OSP-Next: Efficient High-Quality Video Generation with Sparse Sequence Parallelism, HiF8 Quantization, and Reinforcement Learning

- 采用稀疏注意力与序列并行结合,降低计算开销。
- 单卡提速1.64倍,八卡提速1.52倍,支持跨硬件部署。
- 8位量化下性能仅降0.4%,适合资源受限场景使用。
扩散Transformer虽能生成高质量视频,但全注意力的二次方复杂度限制了效率。本文提出OSP-Next,融合稀疏注意力、并行计算、量化与强化学习。模型采用混合全-稀疏注意力架构,其中稀疏部分由Skiparse-2D Attention实现,通过逐标记与分组稀疏化沿空间维度提升局部性,兼容FlashAttention内核。基于重排的局部等价性,提出稀疏序列并行(SSP),通过单次All-to-All通信切换稀疏模式,较Ulysses SP减少75%通信量。引入HiF8量化实现稳定8位联合训练与稀疏微调,并使用Mix-GRPO后训练优化性能。实验表明,OSP-Next在VBench上达83.73%得分,超越Wan2.1基线。在5秒720P与768P设置下,于NVIDIA H200上实现最高1.64×单卡与1.52×八卡加速;在单块Ascend 950PR上,仅损失0.4%得分即达1.69×与2.27×加速,展现跨平台高效性。
原文摘要 · Abstract (English)
Diffusion Transformers achieve strong video generation quality, but the quadratic cost of full attention limits efficiency. We introduce OSP-Next, an efficient text-to-video generation model that integrates sparse attention, parallelism, quantization, and reinforcement learning. OSP-Next uses a hybrid full-sparse attention architecture, where the sparse component is implemented with Skiparse-2D Attention. This fixed-pattern mechanism applies token-wise and group-wise sparse attention along spatial dimensions, leveraging locality while maintaining native compatibility with FlashAttention kernels. Based on the local equivalence of rearrangement in Skiparse-2D Attention, we further propose Sparse Sequence Parallelism (SSP), which partitions subsequences across ranks and switches sparse patterns through a single All-to-All communication. Compared with Ulysses Sequence Parallelism (SP), SSP provides a native parallel strategy for sparse attention and reduces communication volume by 75%. OSP-Next also incorporates HiF8 quantization to enable stable joint training with 8-bit quantization and sparse fine-tuning, and applies Mix-GRPO post-training to improve the performance of the sparse model. Experiments show that OSP-Next achieves a VBench total score of 83.73%, surpassing the Wan2.1 baseline. Under the 5-second 720P and 5-second 768P settings, OSP-Next achieves up to 1.64$\times$ single-GPU speedup and over 1.52$\times$ eight-GPU speedup on NVIDIA H200 GPUs. In addition, with only a 0.4% drop in VBench total score, OSP-Next-HiF8 achieves 1.69$\times$ and 2.27$\times$ speedups under the two settings on a single Ascend 950PR, demonstrating the efficiency and performance of OSP-Next across hardware platforms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。