用强化学习让AI更懂用户，自动推理并精准调用个人历史信息。
Learning to Reason for Multi-Step Retrieval of Personal Context in Personalized Question Answering
- 通过强化学习动态决定何时、从哪调用用户历史信息
- 在LaMP-QA上平均提升8.8%-12%的个性化问答准确率
- 适合需要深度个性化理解的智能客服与助手场景
个性化问答需兼顾答案准确性与用户背景、偏好和历史上下文的一致性。现有先进方法主要依赖检索增强生成（RAG），通过从用户档案中检索相关条目构建个人化上下文。但这些方法直接使用用户查询检索文档，常导致个性化程度浅层。本文提出PR2（个性化检索增强推理），一种融合推理与个人上下文检索的强化学习框架。PR2学习自适应的检索-推理策略，决定何时检索、从用户档案中获取何种证据，以及如何将其融入中间推理步骤。通过在个性化奖励函数下优化多轮推理轨迹，该框架强化了更符合用户特定偏好和上下文信号的推理路径。在使用三个大语言模型的LaMP-QA基准上的大量实验表明，PR2持续优于强基线，平均相对提升8.8%-12%。
原文摘要 · Abstract (English)
Personalization in Question Answering (QA) requires answers that are both accurate and aligned with users' background, preferences, and historical context. Existing state-of-the-art methods primarily rely on retrieval-augmented generation (RAG) solutions that construct personal context by retrieving relevant items from the user's profile. Existing methods use the user's query directly to retrieve personal documents, and such strategies often lead to surface-level personalization. We propose PR2 (Personalized Retrieval-Augmented Reasoning), a reinforcement learning framework that integrates reasoning and retrieval from personal context for personalization. PR2 learns adaptive retrieval-reasoning policies, determining when to retrieve, what evidence to retrieve from user profiles, and how to incorporate it into intermediate reasoning steps. By optimizing multi-turn reasoning trajectories under a personalized reward function, the framework reinforces reasoning paths that better align with user-specific preferences and contextual signals reflected by the reward model. Extensive experiments on the LaMP-QA benchmark using three LLMs show that PR2 consistently outperforms strong baselines, achieving an average relative improvement of 8.8%-12% in personalized QA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。