通过文档相似性增强点击权重,更准确地纠正排序模型的曝光偏差。
Document Similarity Enhanced IPS Estimation for Unbiased Learning to Rank
- 引入文档相似度信息改进传统逆倾向得分(IPS)方法。
- 在n≥30时,相比现有方法提升约3%的NDCG指标(p<0.05)。
- 适合需要高精度排序且训练数据点击量有限的研究与工程场景。
学习排序(LTR)模型依赖历史用户点击数据进行训练,但存在位置偏差——用户更可能点击排名靠前的文档。为缓解此问题,现有方法采用逆倾向得分(IPS)对点击数据重新加权。本文提出一种新方法IPSsim,考虑低排名文档与高排名相关文档之间的相似性,认为相似的低排名文档也可能相关,从而更合理地估计点击概率。在两个大型公开LTR数据集上,我们测试了不同点击数量和模拟点击设置下的表现。结果表明,在top-n排序中当n≥30时,IPSsim显著优于现有方法;例如n=50时,相比文献中的双重稳健估计器,其在NDCG上实现统计显著的约3%提升(p<0.05)。
原文摘要 · Abstract (English)
Learning to Rank (LTR) models learn from historical user interactions, such as user clicks. However, there is an inherent bias in the clicks of users due to position bias, i.e., users are more likely to click highly-ranked documents than low-ranked documents. To address this bias when training LTR models, many approaches from the literature re-weight the users' click data using Inverse Propensity Scoring (IPS). IPS re-weights the user's clicks proportionately to the position in the historical ranking that a document was placed when it was clicked since low-ranked documents are less likely to be seen by a user. In this paper, we argue that low-ranked documents that are similar to highly-ranked relevant documents are also likely to be relevant. Moreover, accounting for the similarity of low-ranked documents to highly ranked relevant documents when calculating IPS can more effectively mitigate the effects of position bias. Therefore, we propose an extension to IPS, called IPSsim, that takes into consideration the similarity of documents when estimating IPS. We evaluate our IPSsim estimator using two large publicly available LTR datasets under a number of simulated user click settings, and with different numbers of training clicks. Our experiments show that our IPSsim estimator is more effective than the existing IPS estimators for learning an unbiased LTR model, particularly in top-n settings when n >= 30. For example, when n = 50, our IPSsim estimator achieves a statistically significant ~3% improvement (p < 0.05) in terms of NDCG compared to the Doubly Robust estimator from the literature.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。