通过分片注意力加速长视频理解，多卡并行提升效率
APB-V: Accelerating Long-Video Understanding via Sequence-Parallelism-aware Approximate Attention
- 将注意力计算拆分到多卡，减少冗余计算
- 在不压缩视觉特征下实现12.72倍加速，性能几乎无损
- 适合需要处理长视频的多模态模型研究者
长视频推理效率仍是大型多模态模型的关键瓶颈，主要源于预填充阶段的密集计算。现有方法或压缩视觉嵌入，或在单卡上使用稀疏注意力，导致加速有限或性能下降，限制了模型处理更长、更复杂视频的能力。为此，我们提出APB-V，一种面向序列并行的近似注意力框架，可在多卡上加速长视频推理。通过分布式近似注意力，该方法降低计算量并提升并行度，实现无需压缩视觉嵌入的高效处理，从而提升任务性能。系统级优化如负载均衡和融合前向传播进一步释放潜力，在不显著损失性能的前提下，相较FlashAttn、ZigZagRing和APB分别实现12.72x、1.70x和1.18x的加速。代码已开源。
原文摘要 · Abstract (English)
The efficiency of long-video inference remains a critical bottleneck, mainly due to the dense computation in the prefill stage of Large Multimodal Models (LMMs). Existing methods either compress visual embeddings or apply sparse attention on a single GPU, yielding limited acceleration or degraded performance and restricting LMMs from handling longer, more complex videos. To overcome these issues, we propose APB-V, a sequence-parallel framework with optimized attention that accelerates long-video inference across multiple GPUs. By distributing approximate attention, APB-V reduces computation and increases parallelism, enabling efficient processing of more visual embeddings without compression and thereby improving task performance. System-level optimizations, such as load balancing and fused forward passes, further unleash the potential of APB-V, delivering speedups of 12.72x, 1.70x, and 1.18x over FlashAttn, ZigZagRing, and APB, without notable performance loss. Code available at https://github.com/thunlp/APB
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。