用因果模型识别用户真实偏好,让大模型更精准个性化。
NextQuill: Causal Preference Modeling for Enhancing LLM Personalization
- 从因果视角建模用户历史对生成内容的影响
- 在多个基准上显著提升个性化效果
- 适合需要深度个性化的大模型应用开发者
将大语言模型(LLM)个性化应用于日常场景日益重要,但现有方法难以区分哪些预测和数据真正反映用户偏好,导致个性化流于表面。本文提出NextQuill,一种基于因果偏好建模的个性化对齐框架。该方法将模型输出与真实数据生成视为受用户偏好及其他因素影响的结果,通过因果干预估计用户历史对每个词元预测或生成实例的真实偏好效应。在此基础上,提出两种互补策略:一是对齐模型内部因果偏好效应与真实数据中的偏好效应,而非盲目拟合输出;二是聚焦由真实数据识别出的承载偏好的词元进行拟合,而非平均对待所有词元。通过融合这两项策略,NextQuill实现从因果偏好效应中学习,推动更有效、更个性化的适配。多个人性化基准实验表明,NextQuill显著提升了个性化质量,为大模型个性化提供了原则性、因果性的基础。代码已开源。
原文摘要 · Abstract (English)
Personalizing large language models (LLMs) for individual users has become increasingly important as they are progressively integrated into real-world applications to support users' daily lives. However, existing personalization approaches often fail to distinguish which components of model predictions and training data truly reflect user preferences, leading to superficial personalization alignment. In this paper, we introduce NextQuill, a novel LLM personalization alignment framework grounded in causal preference modeling. We approach personalization from a causal perspective, treating both model predictions and ground-truth data generation as outcomes influenced by user preferences, along with other factors. We define the true preference effect as the causal impact of user history (which reflects preferences) on each token prediction or data generation instance, estimated through causal intervention techniques. Building on this insight, NextQuill introduces two complementary alignment strategies: (1) aligning model-internal causal preference effects on predictions with those reflected in ground-truth data, rather than indiscriminately fitting predictions, and (2) focusing on fitting preference-bearing tokens identified via ground-truth data preference effects, rather than treating all tokens uniformly. By integrating these strategies, NextQuill shifts the alignment process toward learning from causal preference effects, facilitating more effective and personalized adaptation. Experiments across multiple personalization benchmarks demonstrate that NextQuill significantly improves personalization quality, offering a principled, causal foundation for LLM personalization. Our codes are available on https://github.com/juntaoyou/NextQuill.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。