用强化学习让大模型从单轮问题中推断用户隐含意图,生成更精准的个性化回答。
Training LLMs with Reinforcement Learning for Intent-Aware Personalized Question Answering

- 通过标签化架构在推理中显式建模用户隐含意图
- 在LaMP-QA上平均提升7.5%的宏得分
- 适合需要理解用户深层需求的单轮对话场景
有效的个性化问答需要将回复基于用户的潜在意图(即问题背后的隐含‘为什么’),而现有方法依赖多轮对话上下文或丰富用户画像,在单轮场景下无法显式建模意图。为弥补这一差距,我们提出IAP(意图感知个性化)框架,利用强化学习训练模型从单轮问题中直接推断隐含意图,并通过标签化方案将其融入推理过程,生成意图驱动的个性化回答。通过在个性化奖励函数下优化意图感知的回答轨迹,IAP强化了使隐含意图显式化的生成路径,使回应更贴合用户真实目标。在六个模型上的LaMP-QA基准测试中,IAP持续优于所有基线,平均宏得分提升约7.5%,证明在训练目标中显式建模隐含意图是个性化问答的可行方向。
原文摘要 · Abstract (English)
Effective personalized question answering (PQA) in language models requires grounding responses in the user's underlying intent, where intent refers to the implicit ``why'' behind a query beyond its explicit wording. However, existing approaches to intent-aware personalization rely on multi-turn conversational context or rich user profiles, and do not explicitly model user intent during the reasoning process. This limits their effectiveness in single-turn settings, where the user's latent goal must be inferred from minimal input and integrated into the thinking and reasoning process. To bridge this gap, we propose IAP (Intent-Aware Personalization), a reinforcement learning framework that trains models to infer implicit user intent directly from a single-turn question and incorporate it into thinking steps through a tag-based schema for generating personalized, intent-grounded answers. By optimizing intent-aware answer trajectories under a personalized reward function, IAP reinforces generation paths that make implicit user intent explicit and produce responses that better align with the user's underlying goal. Through experiments on the LaMP-QA benchmark across six models, IAP consistently outperforms all baselines, achieving an average macro-score gain of around 7.5\% over the strongest competitor, demonstrating that modeling implicit user intent within the training objective is a promising direction for PQA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。