用向量建模用户偏好，让聊天机器人记住你，不用重说。
User Preference Modeling for Conversational LLM Agents: Weak Rewards from Retrieval-Augmented Interaction
- 用长短时双向量表示用户偏好，在线更新弱奖励信号。
- 相比无偏好模型，交互效率提升，超时率降低37%。
- 无需微调，适合个性化对话系统研发者使用。
大型语言模型越来越多地被用作个人助手，但大多数缺乏持久的用户模型，导致用户在不同会话中需重复说明偏好。我们提出向量自适应检索评分（VARS），一种与管道无关、冻结主干的框架，将每位用户以长期和短期向量表示在共享偏好空间中，并利用这些向量偏置对结构化偏好记忆的检索评分。向量通过用户反馈产生的弱标量奖励在线更新，实现个性化而无需对每个用户进行微调。我们在 extsc{MultiSessionCollab} 上评估，这是一个具有丰富用户偏好信息的在线多会话协作基准，涵盖数学与代码任务。在冻结主干条件下，用户感知检索的主要优势在于提升交互效率而非显著提高原始任务准确率：我们的完整 VARS 代理达到最优综合表现，任务成功率媲美强基线 Reflection，同时降低超时率与用户操作负担。学习到的长期向量与跨用户偏好重叠一致，短期向量捕捉会话特异性适应，支持双向量设计的可解释性。代码、模型与数据已开源于 https://github.com/YurenHao0426/VARS。
原文摘要 · Abstract (English)
Large language models are increasingly used as personal assistants, yet most lack a persistent user model, forcing users to repeatedly restate preferences across sessions. We propose Vector-Adapted Retrieval Scoring (VARS), a pipeline-agnostic, frozen-backbone framework that represents each user with long-term and short-term vectors in a shared preference space and uses these vectors to bias retrieval scoring over structured preference memory. The vectors are updated online from weak scalar rewards from users' feedback, enabling personalization without per-user fine-tuning. We evaluate on \textsc{MultiSessionCollab}, an online multi-session collaboration benchmark with rich user preference profiles, across math and code tasks. Under frozen backbones, the main benefit of user-aware retrieval is improved interaction efficiency rather than large gains in raw task accuracy: our full VARS agent achieves the strongest overall performance, matches a strong Reflection baseline in task success, and reduces timeout rate and user effort. The learned long-term vectors also align with cross-user preference overlap, while short-term vectors capture session-specific adaptation, supporting the interpretability of the dual-vector design. Code, model, and data are available at https://github.com/YurenHao0426/VARS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。