用结构化场景图提升机器人长任务推理能力
RoBoSR: Structured Scene Representations for Embodied Robotic Reasoning
- 构建物体中心的场景图，实现状态演化的分步建模
- 在零样本泛化和长时任务中超越提示法与传统方法
- 适合需要因果推理的开放世界机器人任务
尽管进展迅速，真实世界中的具身推理仍面临挑战。现有方法依赖示范驱动的序列偏差，限制了在开放、长周期任务中对动态状态进行结构化推理的能力。本文提出RoBoSR，一种中间结构化表示，将操作建模为语义锚定的物体中心场景图中的逐步状态转移。通过在感知-动作界面建模物体状态及其空间关系，该表示将高层任务推理与原始输入解耦，支持对先决条件、效应和目标状态的结构化推理。该表示赋予智能体因果推理能力，强制子任务依赖性，并支持连贯的长周期任务规划。为学习此类结构感知推理，我们构建了Manip-Cognition-1.6M数据集，联合监督跨多样化任务的场景理解、指令解析与子任务规划。在多个基准测试与真实世界演示中，本方法在零样本泛化与长周期任务中持续优于提示法与经典TAMP基线。结果表明，结构化中间表示是可扩展具身推理的关键归纳偏置。
原文摘要 · Abstract (English)
Despite rapid progress, embodied reasoning under real-world variability remains challenging. Existing approaches rely on demonstration-driven sequential biases, limiting flexibility in open-ended and long-horizon tasks that require structured reasoning over evolving states. We introduce RoBoSR, an intermediate structural representation that formulates manipulation as step-wise state transitions over semantically grounded, object-centric scene graphs. By modeling object states and their spatial relations at the perception-action interface, RoBoSR disentangles high-level task reasoning from raw inputs and enables structured reasoning over preconditions, effects, and goal states. This representation endows the agent with causal reasoning capability, enforcing subtask dependencies and supporting coherent long-horizon task planning. To learn such structure-aware reasoning, we construct Manip-Cognition-1.6M, an open-world dataset that jointly supervises scene understanding, instruction interpretation, and subtask planning across diverse tasks. Across several benchmarks and real-world demonstrations, our method consistently outperforms prompting-based methods and classical TAMP baselines in zero-shot generalization and long-horizon tasks. The results underscore structured intermediate representations as a critical inductive bias for scalable embodied reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。