用分解轨迹推理偏好，让AI更懂个性化需求
PREDICT: Preference Reasoning by Evaluating Decomposed preferences Inferred from Candidate Trajectories
- 将用户行为分解为多个偏好成分，逐轮优化
- 在两个环境中比基线提升超40%以上
- 适合需要精准理解用户偏好的智能系统
实现个性化交互的AI代理必须准确理解人类偏好。现有方法虽能利用大模型从用户行为中推断偏好，但常生成泛化、笼统的结果，难以捕捉个体差异。本文提出PREDICT，通过三步增强：（1）对推断偏好进行迭代优化；（2）将偏好拆解为构成组件；（3）在多条轨迹上验证偏好。我们在网格世界和新构建的文本域环境PLUME上评估该方法，结果表明其在捕捉细微偏好方面显著优于基线，在网格世界中提升66.2%，在PLUME中提升41.0%。
原文摘要 · Abstract (English)
Accommodating human preferences is essential for creating AI agents that deliver personalized and effective interactions. Recent work has shown the potential for LLMs to infer preferences from user interactions, but they often produce broad and generic preferences, failing to capture the unique and individualized nature of human preferences. This paper introduces PREDICT, a method designed to enhance the precision and adaptability of inferring preferences. PREDICT incorporates three key elements: (1) iterative refinement of inferred preferences, (2) decomposition of preferences into constituent components, and (3) validation of preferences across multiple trajectories. We evaluate PREDICT on two distinct environments: a gridworld setting and a new text-domain environment (PLUME). PREDICT more accurately infers nuanced human preferences improving over existing baselines by 66.2\% (gridworld environment) and 41.0\% (PLUME).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。