通过分块门控机制压缩视觉语言模型的内存，提升推理速度。
KV-Efficient VLA: A Method to Speed up Vision Language Models with RNN-Gated Chunked KV Cache
- 将KV缓存分块并用循环门控筛选高价值历史信息
- 平均节省24.6%计算量，推理速度提升1.34倍，内存减少1.87倍
- 无需改动控制逻辑，适合实时机器人任务部署
视觉-语言-动作（VLA）模型为机器人感知与控制提供统一框架，但其在长时序真实任务中的扩展受限于注意力计算开销大及推理时需存储大量关键值（KV）对所导致的高内存消耗，尤其当保留历史图像令牌作为上下文时更为明显。现有方法多聚焦于扩大骨干网络以增强泛化能力，较少关注对实时应用至关重要的推理效率问题。本文提出KV-Efficient VLA，一种与模型无关的内存压缩方法，通过引入轻量级机制选择性保留高实用性的上下文。该方法将KV缓存划分为固定大小的块，并使用循环门控模块根据学习到的效用分数总结和过滤历史上下文。此设计旨在保留近期细粒度细节的同时，大幅删除过时且低相关性的记忆内容。实验表明，该方法可实现平均24.6%的浮点运算量（FLOPs）节省、1.34倍的推理加速以及1.87倍的KV内存减少。该方法可无缝集成至现有VLA架构中，实现无修改的可扩展推理。
原文摘要 · Abstract (English)
Vision-Language-Action (VLA) models offer a unified framework for robotic perception and control, but their ability to scale to real-world, long-horizon tasks is limited by the high computational cost of attention and the large memory required for storing key-value (KV) pairs during inference, particularly when retaining historical image tokens as context. Recent methods have focused on scaling backbone architectures to improve generalization, with less emphasis on addressing inference inefficiencies essential for real-time use. In this work, we present KV-Efficient VLA, a model-agnostic memory compression approach designed to address these limitations by introducing a lightweight mechanism to selectively retain high-utility context. Our method partitions the KV cache into fixed-size chunks and employs a recurrent gating module to summarize and filter the historical context according to learned utility scores. This design aims to preserve recent fine-grained detail while aggressively pruning stale, low-relevance memory. Based on experiments, our approach can yield an average of 24.6% FLOPs savings, 1.34x inference speedup, and 1.87x reduction in KV memory. Our method integrates seamlessly into recent VLA stacks, enabling scalable inference without modifying downstream control logic.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。