用深度信息分区域压缩视觉令牌,提速机器人推理且不损精度
DepthCache: Depth-Guided Training-Free Visual Token Merging for Vision-Language-Action Model Inference
- 根据深度划分区域,近处保细节、远处高效压缩
- 跨帧分布合并操作,降低每步计算量1.28倍
- 无需训练修改模型,适合各类视觉语言动作系统
视觉-语言-动作(VLA)模型虽能实现通用机器人操作,但推理延迟高,源于大语言模型处理海量视觉令牌。现有方法均匀剪枝或合并令牌,损害空间推理能力。本文提出DepthCache,一种无需训练的框架,利用深度作为结构先验进行视觉令牌压缩。它将观测按深度分块,实施差异化的合并比例,在保留近场操作区的同时压缩远场背景。通过在连续帧间分配合并过程,利用时间冗余,保证表征一致性并降低单步计算。此外,基于末端执行器动态的运动自适应管道进一步优化辅助视角压缩。该框架无需修改模型,可泛化至多种VLA架构。在LIBERO基准上,DepthCache实现最高1.28倍推理加速,平均成功率下降不足1%,而剪枝与合并基线在相近压缩率下导致4–24%性能下降。真实机器人实验表明,DepthCache显著提升任务吞吐量和闭环控制响应速度。
原文摘要 · Abstract (English)
Vision-Language-Action (VLA) models enable generalist robotic manipulation but suffer from high inference latency. This bottleneck stems from the massive number of visual tokens processed by large language backbones. Existing methods either prune or merge tokens uniformly, degrading the spatial reasoning essential for robotic control. We present DepthCache, a training-free framework that leverages depth as a structural prior for visual token compression. It partitions observations into depth-based regions and applies spatially differentiated merge ratios, preserving the near-field workspace while compressing the distant background. To exploit temporal redundancy, DepthCache distributes the merging process across consecutive frames, ensuring consistent representations while reducing per-step computation. A motion-adaptive pipeline further optimizes auxiliary view compression based on end-effector dynamics. The framework requires no model modification, generalizing across diverse VLA architectures. On the LIBERO benchmark, DepthCache achieves up to 1.28x inference speedup with less than 1% average success rate degradation across three VLA models (pi_0.5, OpenVLA, GR00T), whereas pruning and merging baselines incur 4--24% degradation at comparable compression. Real-world experiments on a physical manipulator demonstrate that DepthCache enables faster task throughput and more responsive closed-loop control in latency-sensitive scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。