用世界模型提升智能体规划能力，让AI更懂环境变化。
World Modeling Makes a Better Planner: Dual Preference Optimization for Embodied Task Planning
- 通过双偏好优化联合学习状态预测与动作选择
- 在VoTa-Bench上任务成功率超越GPT-4o等基线方法
- 自动搜索生成数据，无需人工标注
大型视觉语言模型（LVLMs）在具身任务规划中展现潜力，但仍面临依赖约束和效率不足等挑战。现有方法或仅优化动作选择，或仅在推理时使用世界模型，忽略了建模世界本身对规划的增益。我们提出双重偏好优化（D²PO）框架，通过偏好学习联合优化状态预测与动作选择，使LVLM理解环境动态以提升规划能力。为避免人工标注，我们引入树搜索机制，通过试错实现广泛探索，自动收集轨迹与逐步偏好数据。在VoTa-Bench上的大量实验表明，基于D²PO的方法在Qwen2-VL（7B）、LLaVA-1.6（7B）和LLaMA-3.2（11B）上均显著优于现有方法及GPT-4o，实现了更高任务成功率与更高效的执行路径。
原文摘要 · Abstract (English)
Recent advances in large vision-language models (LVLMs) have shown promise for embodied task planning, yet they struggle with fundamental challenges like dependency constraints and efficiency. Existing approaches either solely optimize action selection or leverage world models during inference, overlooking the benefits of learning to model the world as a way to enhance planning capabilities. We propose Dual Preference Optimization (D$^2$PO), a new learning framework that jointly optimizes state prediction and action selection through preference learning, enabling LVLMs to understand environment dynamics for better planning. To automatically collect trajectories and stepwise preference data without human annotation, we introduce a tree search mechanism for extensive exploration via trial-and-error. Extensive experiments on VoTa-Bench demonstrate that our D$^2$PO-based method significantly outperforms existing methods and GPT-4o when applied to Qwen2-VL (7B), LLaVA-1.6 (7B), and LLaMA-3.2 (11B), achieving superior task success rates with more efficient execution paths.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。