arXiv:2606.06453cs.AI2026-06

Vortex让稀疏注意力算法快速部署,显著提升大模型生成速度。

Vortex: Efficient and Programmable Sparse Attention Serving for AI Agents

论文配图:Vortex: Efficient and Programmable Sparse Attention Serving for AI Agents
图 1 · 摘自论文原文
  • 用可编程前端+页级张量抽象,灵活表达各类稀疏注意力机制。
  • 在NVIDIA B200上实现最高3.46倍吞吐提升,精度不降。
  • 适合研究者和AI Agent快速迭代稀疏注意力设计,尤其适配超大规模模型。

随着生成长度增长,稀疏注意力在大语言模型服务中日益重要。然而,大规模部署和评估新稀疏注意力算法仍需大量工程工作,阻碍了研究人员和AI Agent的探索。为此,我们提出Vortex系统:基于页级张量抽象的可编程前端,支持多种稀疏注意力算法的表达;与现代LLM服务栈紧密集成的高效后端。Vortex实现了稀疏注意力算法的快速原型设计、部署与评估,将理论效率优势转化为实际吞吐提升。结果表明,AI Agent利用Vortex自动生成并优化算法,最佳方案较全连接注意力提升达3.46倍吞吐,且保持精度。同时,Vortex扩展至新兴架构与超大规模模型,在MLA-based GLM-4.7-Flash上实现最高4.7倍吞吐提升,在229B参数的MiniMax-M2.7上实现1.37倍提升,均基于NVIDIA B200 GPU。

原文摘要 · Abstract (English)

Sparse attention is becoming increasingly important for serving large language models (LLMs) as generation lengths continue to grow. However, deploying and evaluating new sparse attention algorithms at scale remains highly engineering-intensive, slowing both human researchers and AI agents in exploring the sparse attention design. To address this challenge, we present Vortex, a system that combines a Python-embedded frontend language atop a page-centric tensor abstraction for expressing a broad range of sparse attention algorithms, with an efficient backend tightly integrated into modern LLM serving stacks. Vortex enables rapid prototyping, deployment, and evaluation of sparse attention algorithms, effectively translating their theoretical efficiency gains into real-world throughput improvements. As a result, Vortex substantially accelerates the design and iteration of sparse attention algorithms. First, AI agents use Vortex to automatically generate and refine diverse algorithms, the best reaching up to $3.46\times$ higher throughput than full attention while preserving accuracy. Second, Vortex extends sparse attention to emerging architectures and very large models that are otherwise hard to experiment with, reaching up to $4.7\times$ higher throughput on the MLA-based GLM-4.7-Flash and $1.37\times$ on the 229B-parameter MiniMax-M2.7 on NVIDIA B200 GPUs.

稀疏注意力大模型服务高效推理AI Agent

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。