构建隐式用户人格数据集,用强化学习让大模型更懂人。
PersonaMem-v2: Towards Personalized Intelligence via Learning Implicit User Personas and Agentic Memory
- 基于1000次真实对话模拟,挖掘用户隐含偏好。
- 强化微调后模型达53%准确率,超越GPT-5。
- 用2k-token记忆取代32k历史,效率提升16倍。
个性化是推动大模型能力与对齐的关键一步。我们提出PersonaMem-v2,当前最先进的人工智能个性化数据集,模拟了超过300种场景下的1000次真实用户-聊天机器人交互,涵盖20,000+用户偏好和128k token的上下文窗口,其中多数偏好为隐式表达,更贴近现实。基于该数据集,我们研究强化微调如何提升模型在长上下文下的用户理解与个性化推理能力。同时,我们设计了一种代理记忆框架,能持续积累并维护一个单一、人类可读的记忆体。实验表明,前沿大模型在隐式个性化任务中仍表现不佳,准确率仅37-48%;尽管支持长上下文,推理仍是瓶颈。通过强化微调,Qwen3-4B成功超越GPT-5,达到53%准确率。此外,我们的代理记忆框架以仅16倍少的输入token(仅2k token记忆)实现55%准确率,显著优于依赖完整32k对话历史的方法。结果验证了数据集的价值,并证明代理记忆是实现真实世界个性化智能的可行路径。
原文摘要 · Abstract (English)
Personalization is one of the next milestones in advancing AI capability and alignment. We introduce PersonaMem-v2, the state-of-the-art dataset for LLM personalization that simulates 1,000 realistic user-chatbot interactions on 300+ scenarios, 20,000+ user preferences, and 128k-token context windows, where most user preferences are implicitly revealed to reflect real-world interactions. Using this data, we investigate how reinforcement fine-tuning enables a model to improve its long-context reasoning capabilities for user understanding and personalization. We also develop a framework for training an agentic memory system, which maintains a single, human-readable memory that grows with each user over time. In our experiments, frontier LLMs still struggle with implicit personalization, achieving only 37-48% accuracy. While they support long context windows, reasoning remains the bottleneck for implicit personalization tasks. Using reinforcement fine-tuning, we successfully train Qwen3-4B to outperforms GPT-5, reaching 53% accuracy in implicit personalization. Moreover, our agentic memory framework achieves state-of-the-art 55% accuracy while using 16x fewer input tokens, relying on a 2k-token memory instead of full 32k conversation histories. These results underscore the impact of our dataset and demonstrate agentic memory as a scalable path toward real-world personalized intelligence.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。