让排序模型输出可解释的点击率概率，且不损失排序效果
MLPlatt: Simple Calibration Framework for Ranking Models
- 用上下文感知的MLPlatt方法校准排序模型输出
- 在两个数据集上F-ECE提升超10%，且保持原有排序能力
- 适合电商场景中需要精准点击率预估的业务方使用
排序模型广泛应用于电商平台的相关性评估，但通常存在可解释性差、缺乏量纲校准的问题，尤其在使用典型排序损失函数训练时更为突出。本文提出一种后处理校准方法MLPlatt，能够保留项目排序关系的同时，将排序器输出转换为可解释的点击率（CTR）概率，适用于下游任务。该方法具有上下文感知特性，在全局及按特定分类字段（如用户国家或设备类型）划分的子组内均表现良好，这在电商平台的业务视角中尤为重要。在两个数据集上的实验表明，相较于现有方法，MLPlatt在F-ECE（Field Expected Calibration Error）指标上提升超过10%。最重要的是，高质量校准无需牺牲排序性能。
原文摘要 · Abstract (English)
Ranking models are extensively used in e-commerce for relevance estimation. These models often suffer from poor interpretability and no scale calibration, particularly when trained with typical ranking loss functions. This paper addresses the problem of post-hoc calibration of ranking models. We introduce MLPlatt: a simple yet effective ranking model calibration method that preserves the item ordering and converts ranker outputs to interpretable click-through rate (CTR) probabilities usable in downstream tasks. The method is context-aware by design and achieves good calibration metrics globally, and within strata corresponding to different values of a selected categorical field (such as user country or device), which is often important from a business perspective of an E-commerce platform. We demonstrate the superiority of MLPlatt over existing approaches on two datasets, achieving an improvement of over 10\% in F-ECE (Field Expected Calibration Error) compared to other methods. Most importantly, we show that high-quality calibration can be achieved without compromising the ranking quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。