arXiv:2605.31545cs.CL2026-05

让AI评价模型更懂用户偏好,从对话历史中自动学习个性化评分标准。

Preference-Aware Rubric Learning for Personalized Evaluation

论文配图:Preference-Aware Rubric Learning for Personalized Evaluation
图 1 · 摘自论文原文
  • 从用户长期交互记录中学习专属评分规则
  • 能准确识别符合用户偏好的文本输出,效果优于传统方法
  • 适合需要个性化评估的AI助手、内容生成等场景

随着大语言模型从通用助手转向以用户为中心的代理,个性化对齐成为关键挑战,而个性化评估仍是瓶颈。现有方法(如自动指标或大模型评判)难以捕捉嵌入长期交互历史中的主观用户偏好。本文提出三个核心原则:代表性、用户一致性与区分性,并引入「个性化评估即学习」范式,将评估视为可学习问题。提出PARL框架,直接从原始用户历史中推导出感知偏好的评分标准,并通过自验证机制确保与用户偏好一致。该框架结合判别式强化学习目标,对比用户原创回复与个性化模型输出,使学习到的评分标准能精准捕捉用户决策边界。在真实个性化文本生成任务上的实验表明,PARL持续生成高保真度评分标准,能可靠识别对齐用户偏好的输出,在不同用户和任务间具有良好泛化能力,同时稳定捕获风格偏好与细粒度评价模式。代码已开源:https://github.com/SnowCharmQ/PARL。

原文摘要 · Abstract (English)

As Large Language Models (LLMs) evolve from general-purpose assistants to user-centric agents, personalization has become central to aligning model behavior with individual preferences, making the evaluation of personalized alignment a critical bottleneck. Existing evaluation methods-ranging from automatic metrics to LLM-as-a-judge approaches-fail to capture subjective, user-specific preferences embedded in long-term interaction histories. We identify three essential principles for reliable and effective personalized evaluation: Representativeness, User-Consistency, and Discriminativeness. To address these principles, we introduce Personalized Evaluation as Learning, a paradigm that formulates personalized evaluation as a learning problem rather than a static judgment. Under this paradigm, we propose PARL (Preference-Aware Rubric Learning for Personalized Evaluation), a framework that learns to induce preference-aware evaluation rubrics directly from raw user histories and performs a self-validation mechanism to ensure consistency with the user's preferences. PARL integrates rubric induction with a discriminative reinforcement learning objective that contrasts user-authored responses against competitive personalized model outputs, enabling the learned rubrics to capture precise, user-specific decision boundaries. Experiments on real-world personalized text generation tasks show that PARL consistently induces high-fidelity rubrics that reliably identify user-aligned responses and generalize across users and tasks, while capturing stable stylistic preferences and fine-grained evaluative patterns. To ensure reproducibility, our code is available at https://github.com/SnowCharmQ/PARL.

个性化评估大模型用户偏好评分标准

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。