arXiv:2506.03324cs.LG2025-06被引 2

优化推荐系统探索率,动态调整随机推荐比例以提升效果。

Optimization of Epsilon-Greedy Exploration

  • 用贝叶斯损失最小化+梯度下降,自动计算最优探索率。
  • 在不同批次大小下表现稳定,优于传统启发式方法。
  • 适合需要动态调整探索策略的推荐系统场景。

现代推荐系统依赖探索来学习用户对新物品的偏好,通常采用简单的均匀探索策略(如epsilon-greedy),因其与机器学习个性化模型兼容。关键挑战在于探索率的选择——应有多少比例的用户流量接收随机推荐,以及这一比例如何随时间变化。尽管已有多种启发式方法应对探索-利用权衡,但实际约束(如批量更新、波动的用户流量、短时间窗口和最低探索要求)使最优探索率难以确定。本文提出一种基于贝叶斯后悔最小化的原则性框架,通过随机梯度下降(SGD)确定探索调度,并利用模型预测控制(MPC)实现动态调整。在多个推荐数据集上的大量实验表明,各周期批次大小的变化显著影响最优探索策略。所提方法能自动适配具体问题设置,始终匹配或超越最佳启发式方法。

原文摘要 · Abstract (English)

Modern recommendation systems rely on exploration to learn user preferences for new items, typically implementing uniform exploration policies (e.g., epsilon-greedy) due to their simplicity and compatibility with machine learning (ML) personalization models. Within these systems, a crucial consideration is the rate of exploration - what fraction of user traffic should receive random item recommendations and how this should evolve over time. While various heuristics exist for navigating the resulting exploration-exploitation tradeoff, selecting optimal exploration rates is complicated by practical constraints including batched updates, time-varying user traffic, short time horizons, and minimum exploration requirements. In this work, we propose a principled framework for determining the exploration schedule based on directly minimizing Bayesian regret through stochastic gradient descent (SGD), allowing for dynamic exploration rate adjustment via Model-Predictive Control (MPC). Through extensive experiments with recommendation datasets, we demonstrate that variations in the batch size across periods significantly influence the optimal exploration strategy. Our optimization methods automatically calibrate exploration to the specific problem setting, consistently matching or outperforming the best heuristic for each setting.

推荐系统探索策略优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。