用哈希加速长文本生成，效率提升且不丢质量。
DASH-KV: Accelerating Long-Context LLM Inference via Asymmetric KV Cache Hashing
- 将注意力机制转为近似最近邻搜索，用异构哈希压缩键值缓存。
- 在LongBench上实现线性复杂度，生成质量媲美全精度注意力。
- 动态混合精度设计，关键词保留高精度计算，适合长文本场景。
标准注意力机制的二次计算复杂度是大语言模型在长上下文推理中的根本瓶颈。现有键值缓存压缩方法虽缓解内存压力，但常牺牲生成质量，且未解决浮点运算的高开销问题。本文提出DASH-KV，一种创新加速框架，通过非对称深度哈希将注意力重构为近似最近邻搜索。在此范式下，设计异构编码结构，差异化映射查询与键，以反映其精度与重用特性的差异。为平衡效率与精度，进一步引入动态混合精度机制，自适应保留关键令牌的全精度计算。在LongBench上的大量实验表明，DASH-KV显著优于当前最优基线方法，同时保持与全注意力相当的性能，将推理复杂度从O(N²)降至线性O(N)。
原文摘要 · Abstract (English)
The quadratic computational complexity of the standard attention mechanism constitutes a fundamental bottleneck for large language models in long-context inference. While existing KV cache compression methods alleviate memory pressure, they often sacrifice generation quality and fail to address the high overhead of floating-point arithmetic. This paper introduces DASH-KV, an innovative acceleration framework that reformulates attention as approximate nearest-neighbor search via asymmetric deep hashing. Under this paradigm, we design an asymmetric encoding architecture that differentially maps queries and keys to account for their distinctions in precision and reuse characteristics. To balance efficiency and accuracy, we further introduce a dynamic mixed-precision mechanism that adaptively retains full-precision computation for critical tokens. Extensive experiments on LongBench demonstrate that DASH-KV significantly outperforms state-of-the-art baseline methods while matching the performance of full attention, all while reducing inference complexity from O(N^2) to linear O(N).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。