贪心策略在离线评估中表现最佳,暴露了推荐系统评估方法的严重缺陷。
Exploitation Over Exploration: Unmasking the Bias in Linear Bandit Recommender Offline Evaluation
- 用贪心策略替代探索,仍能获得顶尖性能。
- 90%以上数据集上,不探索的模型优于或等同于探索型模型。
- 适合关注推荐系统评估方法可靠性的研究者阅读。
多臂赌博机(MAB)算法广泛应用于需要持续增量学习的推荐系统。其核心在于探索与利用的权衡:在选择已知高收益项目与探索新项目之间进行抉择。在上下文线性赌博机中,这一权衡尤为关键,因多种变体共享相同的线性回归基础,主要差异体现在探索策略上。尽管广泛应用,但对MAB的离线评估日益被发现难以可靠衡量探索行为。本研究对多个线性MAB进行了大规模离线实证比较。惊人的是,在超过90%的不同数据集上,无探索的贪心线性模型始终表现优异,常优于或匹配其探索型对手。该现象进一步通过超参数优化得到验证,配置始终倾向于最小化探索,表明纯利用是这些评估设置下的主导策略。研究揭示了当前离线评估协议在反映真实探索效能方面的重大不足,强调亟需开发更稳健的评估方法,以指导未来交互式学习在推荐系统中的评估框架研究。
原文摘要 · Abstract (English)
Multi-Armed Bandit (MAB) algorithms are widely used in recommender systems that require continuous, incremental learning. A core aspect of MABs is the exploration-exploitation trade-off: choosing between exploiting items likely to be enjoyed and exploring new ones to gather information. In contextual linear bandits, this trade-off is particularly central, as many variants share the same linear regression backbone and differ primarily in their exploration strategies. Despite its prevalent use, offline evaluation of MABs is increasingly recognized for its limitations in reliably assessing exploration behavior. This study conducts an extensive offline empirical comparison of several linear MABs. Strikingly, across over 90% of various datasets, a greedy linear model, with no type of exploration, consistently achieves top-tier performance, often outperforming or matching its exploratory counterparts. This observation is further corroborated by hyperparameter optimization, which consistently favors configurations that minimize exploration, suggesting that pure exploitation is the dominant strategy within these evaluation settings. Our results expose significant inadequacies in offline evaluation protocols for bandits, particularly concerning their capacity to reflect true exploratory efficacy. Consequently, this research underscores the urgent necessity for developing more robust assessment methodologies, guiding future investigations into alternative evaluation frameworks for interactive learning in recommender systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。