arXiv:2606.05336cs.CL2026-06

无需标注数据,自监督生成用户画像实现个性化大模型

Self-supervised User Profile Generation for Personalization

  • 用双向批量排序目标自监督训练用户画像生成器
  • 在LaMP基准上性能媲美有监督方法且无需任务标签
  • 适合需要低成本个性化但缺乏标注数据的场景

个性化大语言模型(LLM)在推荐、搜索、对话和内容生成等场景中面临核心挑战:相同查询需根据不同用户返回不同答案。一种有效方法是将用户交互历史归纳为自然语言形式的个人记忆或画像,并作为前缀添加到提示中以实现个性化。现有方法依赖下游任务的标注奖励进行训练,但此类标注成本高且稀疏。为此,我们提出自监督框架BUMP,无需任何下游标签即可训练画像生成器。给定用户交互历史,使用GRPO训练一个LLM,在双向批量排序目标下生成自由文本画像:一个小模型裁判评估(i)生成的画像作为查询时,能否将该用户的保留交互排在同批次其他用户交互之上;(ii)保留交互作为查询时,能否将该用户的画像排在其他用户画像之上。两个方向均采用多正例NDCG评分并合并为每轮的密集奖励,批次内其他用户自动提供免费负样本,每个训练样本仅依赖原始交互日志即可获得监督信号。在LaMP基准上,BUMP在性能上达到或超过闭源API及依赖标注奖励的先前方法,且训练过程中完全无需任务标签。

原文摘要 · Abstract (English)

Personalizing large language models (LLMs) has become a central challenge as LLMs are deployed across recommendation, search, dialogue, and content generation -- settings where the same query should yield different answers given different users. A promising route is to summarize each user's interaction history into a natural-language memory or profile and prepend it to the prompt to facilitate personalization. Existing methods learn such profile generators with explicit rewards derived from labeled downstream tasks, which are expensive and sparse as they require annotated supervision for every target task. In light of this challenge, we introduce Bidirectional User Modeling via Profiles (BUMP), a self-supervised framework that trains a profile generator without any downstream labels. Specifically, given a user's interaction history, we use GRPO to train an LLM to emit a free-form textual profile under a bidirectional in-batch ranking objective: a small LLM judge measures (i) how well the generated profile, used as a query, ranks the user's own held-out interactions above interactions from other users in the batch, and (ii) how well a held-out interaction, used as a query, ranks the user's own profile above profiles of other users. Both directions are scored with multi-positive NDCG and combined into a dense reward per rollout; other users in the batch supply free negatives, so every training example yields supervision from raw interaction logs alone. Evaluated on the LaMP benchmark, BUMP matches or outperforms closed-source APIs and prior methods relying on labeled rewards, while requiring no task label at training.

个性化自监督用户画像LLM

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。