arXiv:2602.16173cs.AIcs.CL2026-02被引 14

让AI agent通过实时反馈持续学习用户偏好,适应变化的个性化需求。

Learning Personalized Agents from Human Feedback

  • 通过显式记忆和双通道反馈实现在线个性化学习
  • 初始个性化误差降低,偏好突变后适应速度显著提升
  • 适合需要长期陪伴的智能助手、推荐系统等场景

现代AI代理虽强大,却常难以匹配个体用户的独特且动态变化的偏好。现有方法多依赖静态数据集,或在交互历史中训练隐式偏好模型,或用外部记忆编码用户画像,但对新用户及偏好演变适应能力差。本文提出个性化人类反馈框架(PAHF),实现持续个性化:通过三步循环——行动前澄清模糊意图、基于记忆检索偏好执行动作、事后反馈更新记忆以应对偏好漂移。我们设计了四阶段评估协议及两个基准(具身操作与在线购物),量化代理从零学习初始偏好并响应人格转变的能力。理论分析与实证结果表明,结合显式记忆与双反馈通道至关重要:PAHF学习速度明显更快,性能显著优于无记忆及单通道基线,有效降低初始个性化误差,并快速适应偏好变化。

原文摘要 · Abstract (English)

Modern AI agents are powerful but often fail to align with the idiosyncratic, evolving preferences of individual users. Prior approaches typically rely on static datasets, either training implicit preference models on interaction history or encoding user profiles in external memory. However, these approaches struggle with new users and with preferences that change over time. We introduce Personalized Agents from Human Feedback (PAHF), a framework for continual personalization in which agents learn online from live interaction using explicit per-user memory. PAHF operationalizes a three-step loop: (1) seeking pre-action clarification to resolve ambiguity, (2) grounding actions in preferences retrieved from memory, and (3) integrating post-action feedback to update memory when preferences drift. To evaluate this capability, we develop a four-phase protocol and two benchmarks in embodied manipulation and online shopping. These benchmarks quantify an agent's ability to learn initial preferences from scratch and subsequently adapt to persona shifts. Our theoretical analysis and empirical results show that integrating explicit memory with dual feedback channels is critical: PAHF learns substantially faster and consistently outperforms both no-memory and single-channel baselines, reducing initial personalization error and enabling rapid adaptation to preference shifts.

个性化持续学习人机反馈智能体

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。