让大模型学会在真实环境中自主规划行动，通过试错强化学习提升任务完成能力。
Unleashing Embodied Task Planning Ability in LLMs via Reinforcement Learning
- 用无监督强化学习让大模型在环境中自主探索，无需人工标注。
- 在两个挑战性环境中任务完成率分别达97.78%和79.92%，显著优于已有方法。
- 适合研究具身智能、强化学习与大模型结合的开发者使用。
大语言模型在各类任务中表现出色，但在需要持续环境感知和动作生成的具身任务规划场景中仍面临挑战。现有方法基于静态知识生成开环动作脚本，难以学习动作与环境反馈间的因果关系，尤其在部分可观测环境中表现不佳。我们提出Embodied Planner-R1，一种以结果为导向的强化学习框架，使大模型通过少量监督实现自主交互探索。该框架包含三项创新：（1）纯强化学习结合群体滚动，通过并行探索实现环境交互；（2）基于任务完成的稀疏奖励机制；（3）交互式策略优化（IPO），高效利用分组轨迹进行学习。在两个文本型具身规划基准测试中，Embodied Planner-R1在ALFWorld上达到97.78%的任务完成率，在ScienceWorld上达79.92%，远超先前方法；在未见过的环境中仅下降3.66%，展现出强泛化能力。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have demonstrated remarkable capabilities across various tasks, yet they face significant challenges in embodied task planning scenarios that require continuous environmental understanding and action generation. Existing approaches generate open-loop action scripts based on static knowledge, making it difficult to learn causal relationships between actions and environmental feedback, particularly in partially observable environments. We introduce Embodied Planner-R1, a novel outcome-driven reinforcement learning framework that enables LLMs to develop interactive capabilities through autonomous exploration with minimal supervision. Our framework incorporates three key innovations: (1) Without human annotations, we employ pure reinforcement learning with group rollout, incorporating in-environment interaction through parallel exploration; (2) completion-driven sparse reward; and (3) Interactive Policy Optimization (IPO) for efficient learning from grouped trajectories. Across two challenging text-based Embodied planning benchmarks, Embodied Planner-R1 achieves impressive completion rates of 97.78% on ALFWorld and 79.92% on ScienceWorld, surpassing prior methods by a large margin, and suffers only a -3.66% drop in previously unseen environments, evidencing strong generalization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。