arXiv:2609.07306cs.CL2026-09

通过复用路由元数据,大幅减少长文本注意力的计算开销。

RouteRelay: Event-Triggered Cross-Layer Route Reuse for Efficient Dynamic Sparse Attention

  • 跨层复用路由信息,仅重算少数关键路径。
  • 在高变化场景下仍保持99.99%路由召回率。
  • 适合需要高效动态稀疏注意力的长序列模型部署。

动态稀疏注意力通过每层将查询块路由到少量关键块来降低长上下文预填充成本。尽管稀疏注意力内核避免了大部分标记交互,但路由器仍需逐层重建块-块评分矩阵,即使所选路径变化不大。我们提出RouteRelay,一种与路由器无关的方法,仅在深度方向复用路由元数据,同时继续使用当前层的查询、键和值计算注意力。锚点层执行完整路由,中间层重新评分前一层的top-k路径以及一组紧凑的近似候选和随机探测块。当某个候选挑战最弱选中块时,才重新路由该查询行。我们给出了top-k稳定性条件、被遗漏挑战者的概率界,以及行选择性GPU执行设计。在可复现的实验中,RouteRelay在低、中、高跨层漂移下分别仅重路由25.0%、55.4%、78.2%的行,且路由召回率不低于99.99%。在不同路由规模下,其召回率保持100.0%,而评估的完整评分对比例从38.4%降至51.6%(当关键块数从128增至1024时)。其未融合的CPU实现仍慢于密集矩阵乘法,暴露出行压缩与日志更新是主要的内核优化目标。

原文摘要 · Abstract (English)

Dynamic sparse attention reduces long-context prefill cost by routing each query chunk to a small set of key chunks at every Transformer layer. The sparse attention kernel avoids most token interactions, but the router still rebuilds a chunk--chunk score matrix layer after layer, even when the selected routes change little. We introduce RouteRelay, a router-agnostic method that reuses only route metadata across depth while continuing to compute attention with the current layer's queries, keys, and values. Anchor layers perform full routing. Intermediate layers rescore the previous top-$k$ route and a compact sentinel set of near-miss and randomly probed chunks. A query row is rerouted only when a sentinel challenges its weakest selected chunk. We give a top-$k$ stability condition, a probabilistic bound on missed challengers, and a row-selective GPU execution design. In a reproducible empirical evaluation, RouteRelay retains at least 99.99% route recall while rerouting 25.0%, 55.4%, and 78.2% of rows under low, moderate, and high cross-layer drift, respectively. Across routing scales, RouteRelay retains 100.0% recall while evaluating 38.4--51.6% of full-routing score pairs as the key-chunk count grows from 128 to 1024. Its unfused CPU execution remains slower than dense matrix multiplication, exposing row compaction and ledger updates as the main kernel-engineering targets.

稀疏注意力长序列处理路由优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。