将快速执行与慢速推理融合,实现机器人高精度实时操控
Fast-in-Slow: A Dual-System Foundation Model Unifying Fast Manipulation within Slow Reasoning
- 用共享参数将执行模块嵌入大模型,统一快慢系统
- 实测控制频率达117.7赫兹,真实任务成功率提升11%
- 适合需要高速精准操作的机器人研究者使用
通用策略与执行效率是机器人操作中的两大关键挑战。尽管近期基于互联网规模预训练视觉语言模型(VLM)的通用策略具备常识推理能力,但常因执行频率低而受限。受卡尼曼双系统理论启发,现有方法采用分离的系统:由基于VLM的系统2负责高层推理,独立的系统1保证实时控制。然而,这种设计使系统1无法充分利用系统2的预训练知识。本文提出快中之慢(FiS),一种统一的双系统视觉-语言-动作(VLA)模型,通过部分参数共享,将系统1的执行模块嵌入系统2的VLM中。该范式不仅实现系统1的高频执行,还促进推理与执行在单一基础模型内的协同。鉴于两者在FiS-VLA中角色根本不同,我们设计其分别接收异构模态输入并以异步频率运行,支持快速精准操作。为实现两系统协调,提出双感知共训练策略,使系统1具备动作生成能力,同时保留系统2的上下文推理表征。评估显示,FiS-VLA在仿真中成功率比之前最优方法高出8%,在真实任务中提升11%,同时在动作块设置为8时达到117.7赫兹的控制频率。
原文摘要 · Abstract (English)
Generalized policy and execution efficiency constitute the two critical challenges in robotic manipulation. While recent foundation policies benefit from the common-sense reasoning capabilities of internet-scale pretrained vision-language models (VLMs), they often suffer from low execution frequency. To mitigate this dilemma, dual-system approaches, inspired by Kahneman's theory, have been proposed to leverage a VLM-based System 2 model handling high-level reasoning and a separate System 1 action model ensuring real-time control. However, existing designs maintain both systems as separate models, limiting System 1 from fully leveraging the rich pretrained knowledge from the VLM-based System 2. In this work, we propose Fast-in-Slow (FiS), a unified dual-system vision-language-action (VLA) model that embeds the System 1 execution module within the VLM-based System 2 by partially sharing parameters. This innovative paradigm not only enables high-frequency execution in System 1 but also facilitates coordination between the reasoning and execution components within a single foundation model of System 2. Given their fundamentally distinct roles within FiS-VLA, we design the two systems to incorporate heterogeneous modality inputs alongside asynchronous operating frequencies, enabling both fast and precise manipulation. To enable coordination between the two systems, a dual-aware co-training strategy is proposed that equips System 1 with action generation capabilities while preserving System 2's contextual reasoning representation. For evaluation, FiS-VLA outperforms previous state-of-the-art methods by 8% in simulation and 11% in real-world tasks in terms of average success rate, while achieving a 117.7 Hz control frequency with action chunk set to eight. Project web page: fast-in-slow.github.io.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。