用在线偏好优化增强语言模型的具身规划能力，提升智能体决策效率。
MCTS-EP: Empowering Embodied Planning with Online Preference Optimization
- 结合大模型与蒙特卡洛树搜索，通过偏好学习在线优化策略
- 在ALFWorld文本/视觉任务中分别达92%和87%成功率，交互步数减少近50%
- 适合研究具身智能、强化学习与自然语言交互的学者参考
本文提出MCTS-EP，一种将大语言模型（LLM）与蒙特卡洛树搜索（MCTS）结合的在线学习框架，用于训练具身智能体。该框架包含三项核心组件：基于MCTS引导探索以收集偏好数据、高效的多模态推理机制，以及基于偏好优化的迭代训练流程。理论上证明，在损失函数强凸条件下，MCTS-EP的性能界优于传统在线策略算法，并可被形式化为GAIL的搜索增强版本。在多个基准测试中，MCTS-EP表现领先：在ALFWorld中，文本任务成功率达92%，视觉任务达87%；在WebShop中平均奖励达0.81。同时，其平均交互步数从18.7/19.5降至10.2/9.9。代码已公开于GitHub。
原文摘要 · Abstract (English)
This paper introduces MCTS-EP, an online learning framework that combines large language models (LLM) with Monte Carlo Tree Search (MCTS) for training embodied agents. MCTS-EP integrates three key components: MCTS-guided exploration for preference data collection, efficient multi-modal reasoning mechanism, and iterative training pipeline based on preference optimization. We theoretically prove that MCTS-EP achieves better performance bounds than conventional on-policy algorithms when the loss function is strongly convex, and demonstrate that it can be formulated as a search-enhanced variant of GAIL. MCTS-EP achieves state-of-the-art performace across serval benchmarks. In ALFWorld, it achieves 92% and 87% success rates for textual and visual tasks. In WebShop, it reaches an average reward of 0.81. MTCS-EP also reduces average interaction steps from from 18.7/19.5 to 10.2/9.9 steps in visual ALFWorld.Code available at: https://github.com/xuhang-2/Embodied-Agent-Planning
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。