提出抗干扰的排序学习算法,有效应对点击作弊问题。
Online Learning to Rank under Corruption: A Robust Cascading Bandits Approach
- 用中位数均值估计器过滤异常点击数据
- 无干扰时达到最优对数级累积损失,有干扰时损失仅线性增加
- 在真实数据集上显著优于现有方法,提升超90%
在线学习排序(OLTR)研究如何从大量候选项目中推荐一个短排名列表,并根据用户点击反馈优化后续推荐。该问题常被建模为级联强化学习,目标是最大化用户在多个时间步内至少点击一个推荐项的概率。然而,系统易受点击欺诈等干扰行为影响,如机器人或付费点击农场注入虚假反馈,误导学习过程并降低用户体验。本文提出一种鲁棒算法MSUCB,首次将中位数均值估计器引入带干扰的强化学习场景。该估计器在无干扰时表现如普通均值,不牺牲性能;在存在干扰时,通过中位数步骤过滤异常样本,使估计值接近真实值。每轮更新该估计可加速实验中的收敛速度。结果表明,MSUCB在无干扰下实现最优对数级遗憾,在干扰下遗憾仅增加与总干扰量相关的常数项。在真实数据集上的全面实验显示,其性能持续优于现有方法,分别较两种先进方法实现97.35%和91.60%的遗憾降低。
原文摘要 · Abstract (English)
Online learning to rank (OLTR) studies how to recommend a short ranked list of items from a large pool and improves future rankings based on user clicks. This setting is commonly modeled as cascading bandits, where the objective is to maximize the likelihood that the user clicks on at least one of the presented items across as many timesteps as possible. However, such systems are vulnerable to click fraud and other manipulations (i.e., corruption), where bots or paid click farms inject corrupted feedback that misleads the learning process and degrades user experience. In this paper, we propose MSUCB, a robust algorithm that incorporates a novel mean-of-medians estimator, which to our knowledge is applied to bandits with corruption setting for the first time. This estimator behaves like a standard mean in the absence of corruption, so no cost is paid for robustness. Under corruption, the median step filters out outliers and corrupted samples, keeping the estimate close to its true value. Updating this estimate at every round further accelerates empirical convergence in experiments. Hence, MSUCB achieves optimal logarithmic regret in the absence of corruption and degrades gracefully under corruptions, with regret increasing only by an additive term tied to the total corruption. Comprehensive and extensive experiments on real-world datasets further demonstrate that our approach consistently outperforms prior methods while maintaining strong robustness. In particular, it achieves a \(97.35\%\) and a \(91.60\%\) regret improvement over two state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。