让对话模型持续学习用户偏好，解决冷启动和长期个性化难题
Towards Proactive Personalization through Profile Customization for Individual Users in Dialogues
- 将对话拆解为单轮任务，用序列决策建模用户偏好推断
- 在理想与嘈杂场景下均优于基线，跨会话偏好保持一致
- 适合需要长期记忆的个性化对话系统开发者
大型语言模型在交互系统中的应用亟需与个体用户的细微动态偏好深度对齐。现有对齐方法多聚焦通用人类价值观或静态单轮偏好，难以应对长期个性化及初始用户冷启动问题。为此，我们提出 PersonalAgent——一种以用户为中心的终身学习代理，可持续推断并适应用户偏好。PersonalAgent 通过将对话分解为单轮交互，构建并动态优化统一用户画像，将偏好推断建模为序列决策任务。实验表明，PersonalAgent 在理想与噪声对话环境中均显著优于强基线（提示工程与策略优化方法），同时保持跨会话偏好一致性。人工评估进一步验证其能自然、连贯地捕捉用户偏好。研究强调了终身个性化在构建更包容、自适应对话系统中的重要性。代码已公开。
原文摘要 · Abstract (English)
The deployment of Large Language Models (LLMs) in interactive systems necessitates a deep alignment with the nuanced and dynamic preferences of individual users. Current alignment techniques predominantly address universal human values or static, single-turn preferences, thereby failing to address the critical needs of long-term personalization and the initial user cold-start problem. To bridge this gap, we propose PersonalAgent, a novel user-centric lifelong agent designed to continuously infer and adapt to user preferences. PersonalAgent constructs and dynamically refines a unified user profile by decomposing dialogues into single-turn interactions, framing preference inference as a sequential decision-making task. Experiments show that PersonalAgent achieves superior performance over strong prompt-based and policy optimization baselines, not only in idealized but also in noisy conversational contexts, while preserving cross-session preference consistency. Furthermore, human evaluation confirms that PersonalAgent excels at capturing user preferences naturally and coherently. Our findings underscore the importance of lifelong personalization for developing more inclusive and adaptive conversational agents. Our code is available here.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。