用数据驱动方法直接优化真实用户点击率，超越传统排序模型。
RewardRank: Optimizing True Learning-to-Rank Utility
- 构建奖励模型预测任意排序的用户真实收益，再通过可微排列优化。
- 在百度和亚马逊数据集上，显著提升点击率与购买概率等真实指标。
- 无需假设位置偏差，适合追求线上真实效果的推荐系统研发者。
传统排序系统依赖简化的离线代理目标，常忽略位置偏差和物品多样性等因素，导致在线A/B测试中无法提升真实反事实效用（如点击率或购买概率）。我们提出RewardRank，一种数据驱动的学习排序框架，直接最大化反事实效用。该框架首先从日志用户行为中学习奖励模型以预测任意排序的效用，再利用可微软排列操作训练排序器进行优化。为实现严谨可复现评估，我们提出两个基准套件：(i) Parametric Oracle Evaluation（PO-Eval），在Baidu-ULTR数据集上使用开源点击模型作为反事实真值；(ii) LLM-as-User Evaluation（LAU-Eval），在Amazon-KDD-Cup数据集上通过大语言模型模拟真实用户行为。RewardRank在两个基准上均取得最高反事实效用，证明优化经典指标（如NDCG）并非最大化真实用户效用的最优路径。此外，基于Baidu-ULTR真实用户反馈，RewardRank在离线相关性性能上达到新纪录。结果表明，学习排序可重构为纯数据驱动的反事实效用直接优化，无需依赖位置偏差等显式建模假设。
原文摘要 · Abstract (English)
Traditional ranking systems optimize offline proxy objectives that rely on oversimplified assumptions about user behavior, often neglecting factors such as position bias and item diversity. Consequently, these models fail to improve true counterfactual utilities such as such as click-through rate or purchase probability, when evaluated in online A/B tests. We introduce RewardRank, a data-driven learning-to-rank (LTR) framework for counterfactual utility maximization. RewardRank first learns a reward model that predicts the utility of any ranking directly from logged user interactions, and then trains a ranker to maximize this reward using a differentiable soft permutation operator. To enable rigorous and reproducible evaluation, we further propose two benchmark suites: (i) Parametric Oracle Evaluation (PO-Eval), which employs an open-source click model as a counterfactual oracle on the Baidu-ULTR dataset, and (ii) LLM-as-User Evaluation (LAU-Eval), which simulates realistic user behavior via large language models on the Amazon-KDD-Cup dataset. RewardRank achieves the highest counterfactual utility across both benchmarks and demonstrates that optimizing classical metrics such as NDCG is sub-optimal for maximizing true user utility. Finally, using real user feedback from the Baidu-ULTR dataset, RewardRank establishes a new state of the art in offline relevance performance. Overall, our results show that learning-to-rank can be reformulated as direct optimization of counterfactual utility, achieved in a purely data-driven manner without relying on explicit modeling assumptions such as position bias. Our code is available at: $https://github.com/GauravBh1010tt/RewardRank$
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。