用动态内容感知的锚点替代静态锚点，显著降低长序列推理计算量。
Dropping the Anchor: Statistical Context Summarization for Distributed Systems via Pulsar Attention
- 引入轻量级内容感知组件替代传统静态锚点，提升注意力稳定性。
- 在128K token长度下性能超越Star Attention，最高比密集注意力高出4.7%。
- 适合需要高效处理超长文本的分布式大模型应用，如智能客服、文档分析。
大型语言模型在长序列上的推理因自注意力的二次复杂度而计算开销巨大。分布式分块方法如Star Attention通过跨主机分片上下文降低开销，但依赖于为每个主机预附一个静态、内容无关的首块副本。本文提出Pulsar Attention，以两个轻量级、内容感知的组件替代静态锚点：一个小规模注意力汇聚前缀以稳定softmax，以及基于最大IDF启发式选择全局稀有词的紧凑跨块摘要。该方法将每GPU的第1阶段浮点运算量最多降低3.3倍，同时保持与Star Attention相同的键值缓存占用。在使用Llama-3.1-8B-Instruct模型的RULER数据集上，Pulsar Attention在序列长度达128K时表现优于Star Attention，且在多数任务中与密集注意力保持竞争力，任务相关绝对增益最高达4.7%。
原文摘要 · Abstract (English)
Inference with large language models (LLMs) on long sequences is computationally expensive due to the quadratic complexity of self-attention. Distributed blockwise methods such as Star Attention reduce this cost by sharding context across hosts, but rely on prepending a static, content-blind copy of the first block to every host. We propose Pulsar Attention, which replaces the static anchor with two lightweight, content-aware components: a small attention-sink prefix that stabilizes softmax, and compact cross-block summaries built via a Max-IDF heuristic that selects chunks containing globally rare tokens. This reduces the Phase 1 per-GPU FLOPs by up to 3.3x over Star Attention while retaining an identical KV cache footprint. On RULER with Llama-3.1-8B-Instruct, Pulsar Attention outperforms Star Attention at sequence lengths up to 128K tokens and remains competitive with dense attention across most tasks, with task-dependent absolute gains of up to 4.7% over the dense baseline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。