通过分析每个词对个性化的重要性,提升大模型回应用户需求的能力。
Rethinking Personalization in Large Language Models at the Token Level
- 用自对比方法评估输出词对用户信息的依赖程度。
- 在训练中动态强化高个性化词,平均提升超10%,最高达68.04%。
- 适合需要精准用户适配的大模型应用,如客服、助手等场景。
随着大语言模型在多任务上表现优异,对个体用户的个性化输出需求日益增长。个性化通常被看作在基础自然语言任务之上增加一层,要求模型在完成任务的同时满足用户特定需求。从词粒度视角看,响应中的不同词对个性化的贡献程度各异,高相关性词应获得更高关注。然而,准确估计这种个性化程度仍具挑战。为此,我们提出PerContrast,一种通过因果干预估算输出词对用户信息依赖性的自对比方法。基于此机制,我们构建PerCE损失函数,利用自举过程在训练中自适应地加权高个性化度词,使模型能交替估计与优化这些词。多个大模型实验表明,PerCE显著提升个性化性能,额外开销极小,在LongLaMP数据集上平均提升超10%,最高达68.04%,且具备强跨任务与跨场景迁移能力。结果凸显词粒度个性化建模的重要性,确立词感知训练为推进个性化大模型的简单而有效范式。
原文摘要 · Abstract (English)
With large language models (LLMs) now performing strongly across diverse tasks, there is growing demand for them to personalize outputs for individual users. Personalization is typically framed as an additional layer on top of a base NLP task, requiring model responses to meet user-specific needs while still accomplishing the underlying task. From a token-level perspective, different tokens in a response contribute to personalization to varying degrees. Tokens with higher personalization relevance should therefore receive greater emphasis when developing personalized LLMs. However, accurately estimating such personalization degrees remains challenging. To address this challenge, we propose PerContrast, a self-contrast method that estimates each output token's dependence on user-specific information through causal intervention. Building on this mechanism, we develop the PerCE loss, which adaptively upweights tokens with higher estimated personalization degrees during training via a bootstrap procedure, enabling the model to alternate between estimating and optimizing these tokens. Experiments on multiple LLMs demonstrate that PerCE substantially improves personalization performance with minimal additional cost, achieving average gains of over 10% and up to 68.04% on the LongLaMP dataset, along with strong cross-task and cross-scenario transferability. These results highlight the importance of token-level personalization modeling and establish token-aware training as a simple yet effective paradigm for advancing personalized LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。