让机器人更懂指令,精准预测关键操作点,提升执行效率与鲁棒性。
PIVOT-R: Primitive-Driven Waypoint-Aware World Model for Robotic Manipulation
- 基于动作原语和关键点感知建模,聚焦任务相关操作点预测。
- 在SeaWave基准上平均性能提升19.45%,动态环境适应更强。
- 异步分层执行机制使效率提升28倍,仅损失2.9%性能,适合实际部署。
语言引导的机器人操作是一项挑战性任务,要求具身智能体根据抽象用户指令完成多种复杂操作。以往方法对数据进行简单拟合,未能揭示指令与底层可执行动作之间的关系,容易记忆表面模式而非获得可迁移知识,因而对动态环境变化敏感。为此,我们提出一种面向机器人操作的原语驱动、关键点感知世界模型(PIVOT-R),专注于任务相关关键点的预测。PIVOT-R由关键点感知世界模型(WAWM)和轻量级动作预测模块构成,前者实现原语解析与原语驱动的关键点预测,后者负责低层动作解码。此外,我们设计了异步分层执行器(AHE),允许不同模块以不同频率运行,从而减少计算冗余,提升执行效率。PIVOT-R在SeaWave基准上优于当前最优开源模型,四类指令任务平均相对提升19.45%。相较于同步执行版本,采用AHE后执行效率提升28倍,性能仅下降2.9%。结果表明,PIVOT-R显著提升了机器人操作的性能与效率。
原文摘要 · Abstract (English)
Language-guided robotic manipulation is a challenging task that requires an embodied agent to follow abstract user instructions to accomplish various complex manipulation tasks. Previous work trivially fitting the data without revealing the relation between instruction and low-level executable actions, these models are prone to memorizing the surficial pattern of the data instead of acquiring the transferable knowledge, and thus are fragile to dynamic environment changes. To address this issue, we propose a PrIrmitive-driVen waypOinT-aware world model for Robotic manipulation (PIVOT-R) that focuses solely on the prediction of task-relevant waypoints. Specifically, PIVOT-R consists of a Waypoint-aware World Model (WAWM) and a lightweight action prediction module. The former performs primitive action parsing and primitive-driven waypoint prediction, while the latter focuses on decoding low-level actions. Additionally, we also design an asynchronous hierarchical executor (AHE), which can use different execution frequencies for different modules of the model, thereby helping the model reduce computational redundancy and improve model execution efficiency. Our PIVOT-R outperforms state-of-the-art (SoTA) open-source models on the SeaWave benchmark, achieving an average relative improvement of 19.45% across four levels of instruction tasks. Moreover, compared to the synchronously executed PIVOT-R, the execution efficiency of PIVOT-R with AHE is increased by 28-fold, with only a 2.9% drop in performance. These results provide compelling evidence that our PIVOT-R can significantly improve both the performance and efficiency of robotic manipulation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。