提出可泛化且保持单调性的新偏好学习模型，提升小数据下准确性。
Generalizing while preserving monotonicity in comparison-based preference learning models
- 基于扩散先验的线性广义布拉德利-特里模型，支持泛化
- 在有限数据下准确率显著提升，单调性更稳定
- 适合需要可靠排序逻辑的小样本场景
若你告诉模型偏好选项a胜过b，通常期望a的评分上升、b下降——即模型具备单调性。然而，当前广泛应用的基于比较的偏好学习模型（包括大语言模型）大多无法保证此性质。此前唯一被证明单调的模型是广义布拉德利-特里模型，但其无法泛化至未比较的数据。本文提出一类带扩散先验的线性广义布拉德利-特里模型，并识别出保证单调性的充分条件。实验表明，单调性并非普遍成立；而新模型在数据有限时显著提升准确率，兼具泛化与单调性优势。
原文摘要 · Abstract (English)
If you tell a learning model that you prefer an alternative $a$ over another alternative $b$, then you probably expect the model to be monotone, that is, the valuation of $a$ increases, and that of $b$ decreases. Yet, perhaps surprisingly, many widely deployed comparison-based preference learning models, including large language models, fail to have this guarantee. Until now, the only comparison-based preference learning algorithms that were proved to be monotone are the Generalized Bradley-Terry models. Yet, these models are unable to generalize to uncompared data. In this paper, we advance the understanding of the set of models with generalization ability that are monotone. Namely, we propose a new class of Linear Generalized Bradley-Terry models with Diffusion Priors, and identify sufficient conditions on alternatives' embeddings that guarantee monotonicity. Our experiments show that this monotonicity is far from being a general guarantee, and that our new class of generalizing models improves accuracy, especially when the dataset is limited.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。