提出新算法解决特征丰富、奖励变化的推荐问题。
A Practical Algorithm for Feature-Rich, Non-Stationary Bandit Problems
- 用嵌入空间改进估计器，结合汤普森采样实现在线学习。
- 在四个数据集上平均累积后悔降低5.7%，新闻点击率提升12.4%。
- 适合需要动态适应的推荐系统，如个性化内容推送。
上下文老虎机在诸多实际问题中非常有用。本文进一步构建更贴近现实的问题：（1）具有密集臂特征的上下文老虎机，（2）非线性奖励函数，（3）奖励分布随时间变化但相关性保持的推广型相关老虎机。该设定适用于更多应用场景，如推荐任务。为此，我们提出条件耦合上下文C3汤普森采样算法，用于伯努利老虎机。该方法结合改进的Nadaraya-Watson估计器与嵌入空间中的汤普森采样，支持无需重训练的在线学习。实证结果表明，C3在四个OpenML表格数据集上的平均累积后悔比次优算法低5.7%，在Microsoft News Dataset（MIND）上点击率提升12.4%。
原文摘要 · Abstract (English)
Contextual bandits are incredibly useful in many practical problems. We go one step further by devising a more realistic problem that combines: (1) contextual bandits with dense arm features, (2) non-linear reward functions, and (3) a generalization of correlated bandits where reward distributions change over time but the degree of correlation maintains. This formulation lends itself to a wider set of applications such as recommendation tasks. To solve this problem, we introduce conditionally coupled contextual C3 Thompson sampling for Bernoulli bandits. It combines an improved Nadaraya-Watson estimator on an embedding space with Thompson sampling that allows online learning without retraining. Empirical results show that C3 outperforms the next best algorithm by 5.7% lower average cumulative regret on four OpenML tabular datasets as well as demonstrating a 12.4% click lift on Microsoft News Dataset (MIND) compared to other algorithms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。