让视觉语言动作模型实时响应环境变化,不卡顿、不延迟。
VLASH: Real-Time VLAs via Future-State-Aware Asynchronous Inference
- 用前一动作片段预演机器人未来状态,缓解异步推理的时间错位。
- 相比同步推理,反应延迟降低11.8倍,精度全面领先所有异步方法。
- 无需改架构或加开销,适配主流视觉语言动作模型,适合高速任务。
视觉-语言-动作模型(VLAs)在各类机器人任务中能力日益增强。然而,这些模型通常采用同步推理,即机器人需等待模型完成推理后才执行动作,无法在动作执行期间感知或响应环境变化。这不仅造成明显动作停滞,还显著增加反应延迟,从根本上限制了VLAs在动态实时任务中的应用。异步推理通过让机器人同时执行动作和进行推理,有望实现连续低延迟控制。但因机器人与环境在推理期间持续演变,预测与执行时间区间出现时间错位,导致动作严重不稳定;现有异步方法要么牺牲精度,要么引入运行时开销以缓解此问题。本文提出VLASH,一种简单有效的异步VLA推理方法,在不改变架构且无额外运行开销的前提下,实现平滑、准确、快速的反应控制。VLASH通过使用前一动作片段滚动推进机器人状态,弥合预测与执行之间的差距。实验表明,相比同步推理,反应延迟最高降低11.8倍,且在精度上始终优于所有异步基线。结合动作量化后,任务完成速度进一步提升1.5–2.0倍,精度损失极小。更重要的是,它使最先进的模型π₀.₅成功应对高速高精度任务,如打乒乓球和打地鼠游戏,而传统同步推理在此类任务中会失败。代码已公开于https://github.com/mit-han-lab/vlash。
原文摘要 · Abstract (English)
Vision-Language-Action models (VLAs) are becoming increasingly capable across diverse robotic tasks. However, these models are typically deployed under synchronous inference, where the robot waits for model inference to complete before acting, and cannot perceive or respond to environmental changes during action execution. This not only introduces noticeable action stalls, but also significantly increases reaction latency, fundamentally limiting the applicability of VLAs to dynamic, real-time tasks. Asynchronous inference offers a promising solution to achieve continuous and low-latency control by enabling robots to execute actions and perform inference simultaneously. However, because the robot and environment continue to evolve during inference, a temporal misalignment arises between the prediction and execution intervals. This leads to significant action instability, while existing asynchronous methods either degrade accuracy or introduce runtime overhead to mitigate it. We propose VLASH, a simple yet effective method for asynchronous VLA inference that delivers smooth, accurate, and fast reaction control without architectural changes or additional runtime overhead. VLASH leverages the future execution-time state by rolling the robot state forward with the previous action chunk, thereby bridging the gap between prediction and execution. Experiments show that VLASH reduces reaction latency by up to 11.8x compared to synchronous inference and consistently outperforms all asynchronous baselines in accuracy. With action quantization, it further achieves 1.5-2.0x task completion speedup with minimal accuracy loss. Moreover, it empowers state-of-the-art VLAs such as $π_{0.5}$ to handle fast-reaction, high-precision tasks including playing ping-pong and playing whack-a-mole, where traditional synchronous inference fails. Code is available at https://github.com/mit-han-lab/vlash
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。