让智能体从少量示范中学习个人偏好,动态调整规划策略。
Learning to Plan with Personalized Preferences
- 从少数示范中隐式学习用户偏好,并用于指导规划。
- 在涵盖数百种偏好的新基准上,现有方法仍难以满足个性化需求。
- 将学习到的偏好作为中间表示,显著提升个性化规划能力。
将AI智能体有效融入日常生活,需要其理解并适应个体人类偏好,尤其在协作角色中。尽管具身智能研究进展显著,但多数方法采用通用策略,忽视了规划中的个人偏好。本文提出通过少量示范学习偏好,并据此调整规划策略。研究发现,偏好虽由极简示范隐含表达,却可在多种规划场景中泛化。为此,我们构建了包含数百种多样偏好的具身规划基准(Preference-based Planning, PbP),覆盖从原子动作到复杂序列的偏好。对当前主流方法的评估显示,符号化方法虽具备可扩展性,但在生成和执行符合个性化偏好的计划方面仍面临挑战。进一步实验表明,将学习到的偏好作为规划中的中间表示,能显著提升智能体构建个性化计划的能力。该研究确立偏好为自适应规划的关键抽象层,为偏好引导的规划生成与执行开辟新方向。
原文摘要 · Abstract (English)
Effective integration of AI agents into daily life requires them to understand and adapt to individual human preferences, particularly in collaborative roles. Although recent studies on embodied intelligence have advanced significantly, they typically adopt generalized approaches that overlook personal preferences in planning. We address this limitation by developing agents that not only learn preferences from few demonstrations but also learn to adapt their planning strategies based on these preferences. Our research leverages the observation that preferences, though implicitly expressed through minimal demonstrations, can generalize across diverse planning scenarios. To systematically evaluate this hypothesis, we introduce Preference-based Planning (PbP) benchmark, an embodied benchmark featuring hundreds of diverse preferences spanning from atomic actions to complex sequences. Our evaluation of SOTA methods reveals that while symbol-based approaches show promise in scalability, significant challenges remain in learning to generate and execute plans that satisfy personalized preferences. We further demonstrate that incorporating learned preferences as intermediate representations in planning significantly improves the agent's ability to construct personalized plans. These findings establish preferences as a valuable abstraction layer for adaptive planning, opening new directions for research in preference-guided plan generation and execution.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。