为注意力机制设计了近乎最优的压缩表示，大幅降低计算开销。
Nearly Optimal Attention Coresets
- 通过构造小规模核心子集近似注意力输出
- 压缩后误差小于ε，大小仅需√d·e^ρ/ε量级
- 适用于大模型推理加速与内存受限场景
我们研究在小空间内估计注意力机制的问题，并证明其存在近乎最优大小的核心子集。具体而言，对于任意单位范数的键值对 (K,V) ∈ ℝ^d，存在一个大小至多为 O(√d · e^(ρ+o(ρ))/ε) 的子集 (K',V')，使得对所有范数不超过 ρ 的查询 q，均有 ‖Attn(q,K,V) − Attn(q,K',V')‖ ≤ ε。该结果优于此前最佳成果。我们还给出了改进的下界：ε-核心子集的大小至少为 Ω(√d · e^ρ/ε)。
原文摘要 · Abstract (English)
We consider the problem of estimating the Attention mechanism in small space, and prove the existence of coresets for it of nearly optimal size. Specifically, we show that for any set of unit-norm keys and values $(K,V)$ in $\mathbb{R}^d$, there exists a subset $(K',V')$ of size at most $O({\sqrt{d} e^{ρ+o(ρ)}/\varepsilon})$ such that \[ \left\| \operatorname{Attn}(q,K,V)- \operatorname{Attn}(q,K',V') \right\| \le \varepsilon \] simultaneously for all queries whose norm is bounded by $ρ$. This outperforms the best known results for this problem. We also offer an improved lower bound showing that $\varepsilon$-coresets must have size $Ω({\sqrt{d} e^ρ/ε})$.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。