让机器人在未知环境中边探索边规划，自动完成复杂任务。
Integrated Exploration and Sequential Manipulation on Scene Graph with LLM-based Situated Replanning
- 用图结构+大模型动态更新环境认知，边走边学。
- 真实家庭场景中任务成功率91.3%，路程减少36.1%。
- 适合需要自主探索与连续操作的机器人应用。
在部分已知环境中，机器人需结合探索以获取信息和任务规划以高效执行。为此，我们提出基于场景图的探索式序列操作规划框架EPoG。EPoG将基于图的全局规划器与基于大语言模型（LLM）的情境局部规划器结合，利用观测结果和LLM预测持续更新信念图，表示已知与未知物体。通过计算目标图与信念图之间的图编辑操作，生成动作序列，并按时间依赖性和移动成本排序。该方法无缝融合探索与序列操作规划。在46个真实家庭场景及5个长时程日常物品运输任务的消融实验中，EPoG取得91.3%的成功率，平均减少36.1%的行程距离。此外，物理移动机械臂在未知且动态环境中成功执行复杂任务，验证了EPoG在现实应用中的潜力。
原文摘要 · Abstract (English)
In partially known environments, robots must combine exploration to gather information with task planning for efficient execution. To address this challenge, we propose EPoG, an Exploration-based sequential manipulation Planning framework on Scene Graphs. EPoG integrates a graph-based global planner with a Large Language Model (LLM)-based situated local planner, continuously updating a belief graph using observations and LLM predictions to represent known and unknown objects. Action sequences are generated by computing graph edit operations between the goal and belief graphs, ordered by temporal dependencies and movement costs. This approach seamlessly combines exploration and sequential manipulation planning. In ablation studies across 46 realistic household scenes and 5 long-horizon daily object transportation tasks, EPoG achieved a success rate of 91.3%, reducing travel distance by 36.1% on average. Furthermore, a physical mobile manipulator successfully executed complex tasks in unknown and dynamic environments, demonstrating EPoG's potential for real-world applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。