用值空间路由替代键向量，实现内存减半的高效注意力机制
Keyless Attention: Value-Space Routing and Value-Only Caching for Efficient Transformers
- 用专门的值空间路由取代传统键投影，彻底消除键向量存储
- 在多个模型上实现KV缓存内存减少50%，解码吞吐量提升
- 适合大规模部署场景，尤其对长文本生成有显著优化
Transformer架构是现代自然语言处理的基础，但在长文本生成中，键-值（KV）缓存带来巨大内存与带宽开销，成为大规模部署的瓶颈。本文提出Keyless Attention，一种新型注意力机制，将传统的键投影替换为专用的值空间路由投影，完全移除注意力计算中的键表示，形成仅保留值的缓存（Value-Only Cache），使KV缓存内存降低50%，同时提升解码吞吐量。多模型实验表明，Keyless Attention在困惑度和下游任务性能上与标准QKV注意力相当，且持续将KV缓存内存减少50%。此外，其验证损失在最佳周期后下降更慢，表明对过拟合具有更强鲁棒性。消融实验确认专用值空间路由至关重要，其性能优于不替换路由角色的键值共享方法。预训练实验进一步验证了其在工业场景中的可行性。
原文摘要 · Abstract (English)
Transformer architectures form the foundation of modern natural language processing, yet the Key-Value (KV) cache introduces substantial memory and bandwidth overhead during long-context generation, increasingly bottlenecking large-scale deployment. We propose Keyless Attention, a novel attention mechanism that replaces the conventional key projection with a dedicated value-space routing projection, eliminating key representations from the attention computation entirely and yielding a Value-Only Cache that reduces KV-cache memory by 50% while improving decode throughput. Experiments across multiple models and architectures demonstrate that Keyless Attention achieves comparable perplexity and downstream task performance to standard QKV attention, while consistently reducing KV-cache memory by 50%. Furthermore, Keyless Attention exhibits slower validation loss degradation after the best epoch, indicating improved robustness against overfitting. Ablation studies confirm that the dedicated value-space routing projection is critical, with Keyless Attention outperforming KV-sharing methods that eliminate the key cache without replacing its routing role. Experiments in the pretraining regime further confirm the viability of Keyless Attention in industrial settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。