arXiv:2606.15285cs.RO2026-06被引 2

让视觉语言动作模型更快响应,实现毫秒级机器人控制。

Acting While Understanding: Asynchronous Semantic-Action Decoupling for Real-Time Vision-Language-Action Models

论文配图:Acting While Understanding: Asynchronous Semantic-Action Decoupling for Real-Time Vision-Language-Action Models
图 1 · 摘自论文原文
  • 将语义理解与动作生成解耦,分高/低频运行,降低延迟。
  • 在LIBERO数据集上达到35.6赫兹的推理吞吐率,支持高频闭环控制。
  • 无需重训练模型,适合已有视觉语言动作系统的快速升级。

视觉语言动作模型(VLAs)在机器人操作任务中展现出强大的理解与泛化能力,但全模型推理带来的高计算开销限制了其在低延迟、高频率闭环控制中的部署。本文提出一种异步语义-动作解耦框架,不改变现有VLA的视觉-语言主干或引入外部规划器,仅在内部语义-动作接口处进行解耦。低频理解模块异步更新可复用的语义条件,高频动作模块持续输出控制动作,无需重复调用完整模型。为缓解陈旧语义与当前执行状态的时间错位问题,引入历史动作条件和时间错配训练,提供短时执行上下文,增强在语义滞后情况下的反馈控制鲁棒性。在LIBERO数据集上基于$π_{0.5}$和UniVLA的实验,以及使用UniVLA的真实机器人部署结果表明,该框架实现了高达35.6赫兹的服务器端动作模块推理吞吐率,并在不以控制频率运行完整VLA的前提下,提供了低侵入性的高频闭环控制路径。

原文摘要 · Abstract (English)

Vision-Language-Action models (VLAs) have demonstrated strong task understanding and generalization in robotic manipulation, yet the high computational cost of full-model inference limits their deployment in low-latency, high-frequency closed-loop control. We propose an asynchronous semantic-action decoupling framework that separates semantic understanding from action generation along the internal semantic-action interface of existing VLAs, without redesigning the vision-language backbone or introducing an external planner. A low-frequency understanding module asynchronously updates reusable semantic conditions, while a high-frequency action module continuously outputs control actions without repeatedly invoking the full model. To mitigate the temporal mismatch between stale semantics and the current execution state, we further introduce historical action conditioning and time-misalignment training, which provide short-horizon execution context and improve feedback control robustness under stale semantic conditions. Experiments on LIBERO with $π_{0.5}$ and UniVLA, together with real-robot deployment using UniVLA, show that the proposed framework achieves up to 35.6 Hz server-side action-module inference throughput and offers a low-intrusion path to high-frequency closed-loop control without running full VLA inference at control rate.

机器人控制视觉语言实时系统模型加速

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。