让AI在3D场景中更智能地调整目标,提高任务成功率。
Context Matters! Relaxing Goals with LLMs for Feasible 3D Scene Planning
- 用LLM理解环境并动态调整目标约束,实现分层目标松弛
- 相比现有方法提升52.45%成功率,在真实机器人上验证可行
- 适合需要灵活应对复杂现实环境的具身智能系统
具身智能体需在真实复杂的3D环境中可靠规划与行动。传统规划(如PDDL)虽有结构保障,但在感知噪声和谓词误标注下易失效;而基于大语言模型(LLM)的规划虽具备常识推理能力,却常提出不可行或不安全的动作。受近期融合两类方法的研究启发,本文提出ContextMatters框架,将LLM与经典规划结合,实现分层目标松弛:由LLM协助符号与场景对齐,当目标不可达时,逐步提出功能等价但约束更宽松的新目标,使目标适应当前环境上下文。该机制基于3D场景图运作,将许多名义上不可行的任务转化为可解计划,并支持在无法完全达成时的部分成功。实验表明,相较最先进的LLM+PDDL基线,成功率提升52.45%。此外,我们在TIAGo机器人上部署并验证了该框架的实际可行性。代码、数据集及补充材料已开源。
原文摘要 · Abstract (English)
Embodied agents need to plan and act reliably in real and complex 3D environments. Classical planning (e.g., PDDL) offers structure and guarantees, but in practice it fails under noisy perception and incorrect predicate grounding. On the other hand, Large Language Models (LLMs)-based planners leverage commonsense reasoning, yet frequently propose actions that are unfeasible or unsafe. Following recent works that combine the two approaches, we introduce ContextMatters, a framework that fuses LLMs and classical planning to perform hierarchical goal relaxation: the LLM helps ground symbols to the scene and, when the target is unreachable, it proposes functionally equivalent goals that progressively relax constraints, adapting the goal to the context of the agent's environment. Operating on 3D Scene Graphs, this mechanism turns many nominally unfeasible tasks into tractable plans and enables context-aware partial achievement when full completion is not achievable. Our experimental results show a +52.45% Success Rate improvement over state-of-the-art LLMs+PDDL baseline, demonstrating the effectiveness of our approach. Moreover, we validate the execution of ContextMatter in a real world scenario by deploying it on a TIAGo robot. Code, dataset, and supplementary materials are available to the community at https://lab-rococo-sapienza.github.io/context-matters/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。