提出分层Top-P稀疏注意力，显著降低长文本生成计算开销。
Double-P: Hierarchical Top-P Sparse Attention for Long-Context LLMs
- 分两阶段优化：先聚类粗筛，再按需细筛，动态分配计算资源。
- 在长文本任务中保持接近零误差，计算量最多降1.8倍。
- 适合需要高效推理的长上下文大模型应用，如文档生成、代码补全。
随着长上下文推理成为大语言模型的核心需求，键值缓存上的注意力计算成为主要瓶颈，推动了稀疏注意力的发展。固定预算的Top-K稀疏注意力无法适应不同头和层间注意力分布的异质性，而传统Top-P方法虽能直接保留注意力质量，但未能同时优化精度、选择开销与稀疏计算成本，限制整体效率。本文提出Double-P，一种分层稀疏注意力框架，从簇级使用加权中心点进行粗粒度Top-P估计，再通过第二阶段自适应地仅在必要时分配词元级注意力。在多个长上下文基准测试中，Double-P始终实现近乎零的精度损失，注意力计算开销最多降低1.8倍，并相比最先进固定预算稀疏注意力方法实现最高1.3倍的端到端解码加速。
原文摘要 · Abstract (English)
As long-context inference becomes central to large language models (LLMs), attention over growing key-value caches emerges as a dominant decoding bottleneck, motivating sparse attention for scalable inference. Fixed-budget top-k sparse attention cannot adapt to heterogeneous attention distributions across heads and layers, whereas top-p sparse attention directly preserves attention mass and provides stronger accuracy guarantees. Existing top-p methods, however, fail to jointly optimize top-p accuracy, selection overhead, and sparse attention cost, which limits their overall efficiency. We present Double-P, a hierarchical sparse attention framework that optimizes all three stages. Double-P first performs coarse-grained top-p estimation at the cluster level using size-weighted centroids, then adaptively refines computation through a second top-p stage that allocates token-level attention only when needed. Across long-context benchmarks, Double-P consistently achieves near-zero accuracy drop, reducing attention computation overhead by up to 1.8x and delivers up to 1.3x end-to-end decoding speedup over state-of-the-art fixed-budget sparse attention methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。