让大模型显式推理用户画像，提升个性化生成效果
PrLM: Learning Explicit Reasoning for Personalized RAG via Contrastive Reward Optimization
- 用对比奖励模型训练大模型显式分析用户画像
- 在三个数据集上超越现有方法，对检索结果不敏感
- 适合需要精准个性化的对话与内容生成场景
个性化检索增强生成（RAG）旨在通过结合用户画像和输入查询，生成符合用户偏好的回答。现有方法主要依赖大型语言模型隐式融合检索到的上下文，但对检索质量敏感，易产生与用户偏好不符的输出。为此，我们提出PrLM，一种基于对比奖励优化的强化学习框架，使大模型能够显式推理已检索的用户画像。该框架无需标注推理路径，即可从用户反馈中学习。在三个个性化文本生成数据集上的实验表明，PrLM优于现有方法，并在不同检索数量和不同检索器条件下均保持鲁棒性。
原文摘要 · Abstract (English)
Personalized retrieval-augmented generation (RAG) aims to produce user-tailored responses by incorporating retrieved user profiles alongside the input query. Existing methods primarily focus on improving retrieval and rely on large language models (LLMs) to implicitly integrate the retrieved context with the query. However, such models are often sensitive to retrieval quality and may generate responses that are misaligned with user preferences. To address this limitation, we propose PrLM, a reinforcement learning framework that trains LLMs to explicitly reason over retrieved user profiles. Guided by a contrastively trained personalization reward model, PrLM effectively learns from user responses without requiring annotated reasoning paths. Experiments on three personalized text generation datasets show that PrLM outperforms existing methods and remains robust across varying numbers of retrieved profiles and different retrievers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。