让机器人视觉语言动作策略异步运行,提升全身体操作的实时性与稳定性。
Asynchronous Fast-Slow Vision-Language-Action Policies for Whole-Body Robotic Manipulation
- 分快慢两条路径:快路高频生成动作,慢路用大模型做语义推理,通过隐空间缓存衔接
- 30赫兹全身体动作生成,速度是同类模型的3倍,支持30亿参数大模型
- 适合需要高响应、多关节协同的工业机器人场景,已集成至商用平台
多数视觉-语言-动作(VLA)系统将视觉语言模型(VLM)与动作专家同步运行,但受限于大VLM的低推理速度,整体性能受制。这种强制同步严重制约了全身体操作在多关节、大运动空间和动态视图下的控制稳定性和实时性。本文提出真正异步的快-慢VLA框架(DuoCore-FS),将系统分为高速动作生成路径与低速语义推理路径。关键特征包括:1)使用隐空间表示缓存,存储指令语义与动作-推理表征,对快速路径提供高层引导;2)引入全身体动作标记器,实现统一紧凑的动作表示。尽管采用异步执行,仍保持端到端联合训练,维持政策一致性。该框架在支持30亿参数的VLM基础上,实现30赫兹的全身体动作块生成,约是同等规模模型的三倍速度。真实世界实验显示任务成功率更高,响应性显著增强。DuoCore-FS的训练、推理与部署代码已由Astribot开源,集成于其机器人平台。
原文摘要 · Abstract (English)
Most Vision-Language-Action (VLA) systems integrate a Vision-Language Model (VLM) for semantic reasoning with an action expert generating continuous action signals, yet both typically run at a single unified frequency. As a result, policy performance is constrained by the low inference speed of large VLMs. This mandatory synchronous execution severely limits control stability and real-time performance in whole-body robotic manipulation, which involves more joints, larger motion spaces, and dynamically changing views. We introduce a truly asynchronous Fast-Slow VLA framework (DuoCore-FS), organizing the system into a fast pathway for high-frequency action generation and a slow pathway for rich VLM reasoning. The system is characterized by two key features. First, a latent representation buffer bridges the slow and fast systems. It stores instruction semantics and action-reasoning representation aligned with the scene-instruction context, providing high-level guidance to the fast pathway. Second, a whole-body action tokenizer provides a compact, unified representation of whole-body actions. Importantly, the VLM and action expert are still jointly trained end-to-end, preserving unified policy learning while enabling asynchronous execution. DuoCore-FS supports a 3B-parameter VLM while achieving 30 Hz whole-body action-chunk generation, approximately three times as fast as prior VLA models with comparable model sizes. Real-world whole-body manipulation experiments demonstrate improved task success rates and significantly enhanced responsiveness compared to synchronous Fast-Slow VLA baselines. The implementation of DuoCore-FS, including training, inference, and deployment, is provided to commercial users by Astribot as part of the Astribot robotic platform.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。