用二值反馈精准建模用户差异，提升大模型个性化能力
Personalizing LLMs with Binary Feedback: A Preference-Corrected Optimization Framework
- 将其他用户数据作为隐式负样本，捕捉个体偏好差异
- 通过去偏机制避免共性知识被误判为错误偏好
- 适用于需个性化的大模型场景，如对话系统、内容生成
大语言模型个性化旨在使模型行为符合个体用户偏好。现有方法多关注孤立用户历史，忽视用户间的本质差异。本文提出C-BPO框架，通过偏好校准的二值信号实现模型个性化：将目标用户数据视为正向反馈，其他用户数据作为辅助的隐式负样本，以捕捉用户间差异。为缓解偏好重叠问题（即共享任务知识被错误惩罚），基于正例-未标记例（PU）学习理论构建优化目标，通过减去“正向偏差”来净化负样本，确保在保留个性特征的同时不损害通用帮助性。在多种个性化任务和主流大模型上进行实验，结果表明C-BPO持续优于基线，验证了偏好校准二值信号在建模用户差异方面的有效性。
原文摘要 · Abstract (English)
Large Language Model (LLM) personalization aims to align model behaviors with individual user preferences. Existing methods often focus on isolated user histories, neglecting the essential role of inter-user differences. We propose C-BPO, a framework that personalizes LLMs via preference-calibrated binary signals. By treating target user data as positive feedback and other users' data as an auxiliary set of implicit negative signals, C-BPO captures distinct inter-user differences. To mitigate the preference overlap issue, where shared task knowledge is erroneously penalized, we derive an objective grounded in Positive-Unlabeled (PU) learning theory. This approach purifies negative signals by subtracting ``positive bias'', ensuring alignment with unique idiosyncrasies without compromising general helpfulness. Empirical experiments across various personalization tasks and backbone LLMs show C-BPO consistently outperforms baselines, demonstrating the efficacy of preference-calibrated binary signals in modeling inter-user differences.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。