两塔模型在真实推荐中表现下降？这篇论文揭秘原因并给出解决方案。
Unidentified and Confounded? Understanding Two-Tower Models for Unbiased Learning to Rank
- 分析两塔模型的可识别性条件，指出需文档位置互换或特征重叠才能准确学习。
- 发现日志策略本身不引入偏差，但模型不准时会放大偏差，尤其当预测误差与位置相关。
- 提出样本加权方法缓解偏差，并为工业界使用两塔模型提供实用建议。
加法型两塔模型是工业场景中处理用户反馈偏差的流行学习排序方法。然而，近期研究发现：在性能良好的生产系统收集的点击数据上训练两塔模型，反而导致排名性能下降。本文探讨了两种可能解释：日志策略引起的混淆效应和模型可识别性问题。理论分析表明，仅当文档在不同位置间发生交换，或特征分布存在重叠时，才能从点击数据中恢复模型参数。同时研究发现，若模型能完美捕捉用户行为，日志策略不会引入偏差；但当模型存在偏差时，日志策略会放大该偏差，特别是当预测误差与文档位置相关时。为此，本文提出一种样本加权技术以减轻此类影响，并为研究人员和从业者提供可操作的实践洞察。
原文摘要 · Abstract (English)
Additive two-tower models are popular learning-to-rank methods for handling biased user feedback in industry settings. Recent studies, however, report a concerning phenomenon: training two-tower models on clicks collected by well-performing production systems leads to decreased ranking performance. This paper investigates two recent explanations for this observation: confounding effects from logging policies and model identifiability issues. We theoretically analyze the identifiability conditions of two-tower models, showing that either document swaps across positions or overlapping feature distributions are required to recover model parameters from clicks. We also investigate the effect of logging policies on two-tower models, finding that they introduce no bias when models perfectly capture user behavior. However, logging policies can amplify biases when models imperfectly capture user behavior, particularly when prediction errors correlate with document placement across positions. We propose a sample weighting technique to mitigate these effects and provide actionable insights for researchers and practitioners using two-tower models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。