用物理仿真数字孪生实现机器人操作规划,更准更稳。
Embodied Tree of Thoughts: Deliberate Manipulation Planning with Embodied World Model
- 通过语义空间分析生成多样执行路径
- 在模拟器中诊断失败并迭代优化计划
- 适合需要长程规划与容错的机器人任务
世界模型已成为机器人操作规划的关键组件,使智能体能在执行前预测环境状态并推理动作后果。尽管视频生成模型被广泛采用,但往往缺乏严格的物理约束,导致幻觉和长期物理约束不一致。为此,我们提出具身思维树(EToT),一种基于物理引擎的实时-仿真-实时规划框架,利用交互式数字孪生作为具身世界模型。EToT将操作规划建模为树搜索,通过两种协同机制扩展:(1) 先验分支,基于语义与空间分析生成多样化候选路径;(2) 反思分支,利用视觉语言模型在模拟器中诊断执行失败,并迭代修正计划。通过将高层推理锚定于物理模拟器,该框架确保生成计划严格遵守刚体动力学与碰撞约束。我们在一系列短时与长时操作任务上验证EToT,结果表明其显著优于基线方法,在预测物理动态和应对潜在失败方面表现优异。
原文摘要 · Abstract (English)
World models have emerged as a pivotal component in robot manipulation planning, enabling agents to predict future environmental states and reason about the consequences of actions before execution. While video-generation models are increasingly adopted, they often lack rigorous physical grounding, leading to hallucinations and a failure to maintain consistency in long-horizon physical constraints. To address these limitations, we propose Embodied Tree of Thoughts (EToT), a novel Real2Sim2Real planning framework that leverages a physics-based interactive digital twin as an embodied world model. EToT formulates manipulation planning as a tree search expanded through two synergistic mechanisms: (1) Priori Branching, which generates diverse candidate execution paths based on semantic and spatial analysis; and (2) Reflective Branching, which utilizes VLMs to diagnose execution failures within the simulator and iteratively refine the planning tree with corrective actions. By grounding high-level reasoning in a physics simulator, our framework ensures that generated plans adhere to rigid-body dynamics and collision constraints. We validate EToT on a suite of short- and long-horizon manipulation tasks, where it consistently outperforms baselines by effectively predicting physical dynamics and adapting to potential failures. Website at https://embodied-tree-of-thoughts.github.io .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。