提出事务注意力机制，让关键密钥信息在压缩时不被误删。
Transactional Attention: Semantic Sponsorship for KV-Cache Retention
- 用结构化锚点（如'key:'）标记敏感信息，防止其被缓存淘汰。
- 在16个令牌（0.4%上下文）时仍实现100%凭证召回率，六种基线全为0%。
- 适合需要高安全性的长上下文应用，如函数调用、配置管理场景。
当缓存保留长度K=16（4K上下文的0.4%）时，现有所有KV缓存压缩方法在凭证检索任务上均达到0%准确率。失败原因是‘休眠令牌’：即在训练阶段注意力极低但生成时至关重要的密钥、API密钥和配置值。由于这些令牌缺乏淘汰策略依赖的统计信号，基于注意力分数、重建损失或学习保留门的方法均无法有效保留它们。本文提出事务注意力（Transactional Attention, TA），通过结构化锚点模式（如'key:'、'password:'）对邻近的值型令牌实施‘赞助’保护，防止其被移除。TA在K=16时实现100%凭证召回率，而六种基线方法（H2O、TOVA、SnapKV、StreamingLLM、PyramidKV、DynamicKV）均为0%；并在200次函数调用试验中保持100%准确率。TA-Fast为无注意力变体，内存开销降低52%，兼容SDPA与FlashAttention。TA与现有压缩方法正交，延迟增加不足1%。
原文摘要 · Abstract (English)
At K=16 tokens (0.4% of a 4K context), every existing KV-cache compression method achieves 0% on credential retrieval. The failure mode is dormant tokens: credentials, API keys, and configuration values that receive near-zero attention but become essential at generation time. Because these tokens lack the statistical signals that eviction policies rely on, no method based on attention scores, reconstruction loss, or learned retention gates retains them. We introduce Transactional Attention (TA), a sponsorship mechanism in which structural anchor patterns (e.g., "key:", "password:") protect adjacent value-bearing tokens from eviction. TA achieves 100% credential retrieval at K=16 where six baselines (H2O, TOVA, SnapKV, StreamingLLM, PyramidKV, DynamicKV) achieve 0%, and sustains 100% accuracy across 200 function-calling trials. TA-Fast, an attention-free variant, reduces memory overhead by 52% and is compatible with SDPA and FlashAttention. TA is orthogonal to existing compression methods and adds less than 1% latency overhead.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。