提出混合推测解码框架，加速具身视觉语言动作模型推理。
HeiSD: Hybrid Speculative Decoding for Embodied Vision-Language-Action Models with Kinematic Awareness
- 融合检索与草稿的混合推测解码，提升生成效率。
- 实测在仿真中提速2.45倍，真实场景提速2.06~2.41倍。
- 引入运动学融合度量，自动确定混合边界，适合机器人控制场景。
视觉语言动作（VLA）模型已成为机器人控制的主流方案，但存在推理速度慢的问题。推测解码（SD）是一种有前景的加速方法，可分为基于草稿和基于检索两类。两者在应用于VLA模型时各有优劣，因此我们假设将二者结合可获得更优性能。本文通过详细分析验证了混合使用的可行性。然而，实现混合SD仍面临挑战：(1) 基于检索的SD存在草稿拒绝与持续错误；(2) 混合边界难以确定。为此，我们提出HeiSD框架：引入验证跳过机制与序列级宽松接受策略优化检索式SD；设计基于运动学的融合度量，自动确定混合边界。实验表明，HeiSD在仿真基准上实现最高2.45倍加速，在真实场景中提速2.06~2.41倍，同时保持高任务成功率。
原文摘要 · Abstract (English)
Vision-Language-Action (VLA) Models have become the mainstream solution for robot control, but suffer from slow inference speeds. Speculative Decoding (SD) is a promising acceleration method which can be divided into two categories: drafter-based SD and retrieval-based SD. Each of the two methods demonstrates complementary advantages and limitations when applied to VLA models, leading to the hypothesis that a hybrid approach integrating these two methods will yield better performance. In this paper, we first conduct a series of detailed analyses to reveal the advantages and feasibility of hybrid utilization. However, even with the aforementioned key insights, implementing hybrid SD in VLA models presents several challenges: (1) draft rejection and persistent errors in retrieval-based SD; (2) difficulty in determining the hybrid boundary. To address these, we propose the HeiSD framework. We propose a retrieval-based SD optimization method in HeiSD, which contains a verify-skip mechanism and a sequence-wise relaxed acceptance strategy. Moreover, we proposed a kinematic-based fused metric in HeiSD to automatically determine the hybrid boundary. Experimental results demonstrate that HeiSD attains a speedup of up to 2.45x in simulation benchmarks and 2.06x~2.41x in real-world scenarios, while sustaining a high task success rate.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。