提出跨层注意力聚合方法，显著加速大模型预填充阶段推理
CLAA: Cross-Layer Attention Aggregation for Accelerating LLM Prefill
- 通过聚合多层注意力得分，稳定提升关键词重要性判断
- 相比完整缓存基线，首令牌生成时间缩短最高达39%
- 适用于长上下文场景下需要快速响应的大模型应用
长上下文大模型推理中的预填充阶段仍是计算瓶颈。现有基于词元排序的启发式方法虽能加速推理，但存在各层间词元重要性估计不稳定的问题，且难以独立评估其质量。为此，本文提出答案感知的真值评估器（Answer-Informed Oracle），通过测量生成答案对提示词的注意力反向回溯，定义真实词元重要性。该评估揭示：现有启发式方法在不同层间排名差异显著，部分层级性能骤降，而这一缺陷无法被端到端基准发现。诊断表明，简单将多层得分聚合即可缓解问题。据此提出跨层注意力聚合（CLAA）方法，使性能逼近真值上界，相较完整键值缓存基线，时延降低最高达39%。
原文摘要 · Abstract (English)
The prefill stage in long-context LLM inference remains a computational bottleneck. Recent token-ranking heuristics accelerate inference by selectively processing a subset of semantically relevant tokens. However, existing methods suffer from unstable token importance estimation, often varying between layers. Evaluating token-ranking quality independently from heuristic-specific architectures is challenging. To address this, we introduce an Answer-Informed Oracle, which defines ground-truth token importance by measuring attention from generated answers back to the prompt. This oracle reveals that existing heuristics exhibit high variance across layers: rankings can degrade sharply at specific layers, a failure mode invisible to end-to-end benchmarks. The diagnosis suggests a simple fix: aggregate scores across layers rather than relying on any single one. We implement this as Cross-Layer Attention Aggregation (CLAA), which closes the gap to the oracle upper bound and reduces Time-to-First-Token (TTFT) by up to 39\% compared to the Full KV Cache baseline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。