用机器学习模型提升推荐系统决策效率
A UCB Bandit Algorithm for General ML-Based Estimators
- 将任意机器学习模型融入强化学习框架，通过学习曲线建模解决探索难题
- 理论证明算法达到次线性损失，实验显示比传统方法性能显著提升
- 适合需要高效在线学习的推荐系统、广告投放等场景
我们提出ML-UCB，一种可整合任意机器学习模型的广义上置信界算法。在将复杂机器学习模型用于序列决策时，主要挑战在于缺乏可用于合理探索的可处理集中不等式。本文通过直接建模底层估计器的学习曲线行为克服该限制：假设均方误差随训练样本数按幂律下降，推导出广义集中不等式，并证明ML-UCB实现次线性后悔。该框架允许对任何可通过经验刻画学习曲线的机器学习模型进行有原则的集成，无需针对特定模型进行理论分析。我们在基于在线矩阵分解的协同过滤推荐系统上使用模拟两塔模型的合成数据进行了验证，结果表明其性能显著优于LinUCB。
原文摘要 · Abstract (English)
We present ML-UCB, a generalized upper confidence bound algorithm that integrates arbitrary machine learning models into multi-armed bandit frameworks. A fundamental challenge in deploying sophisticated ML models for sequential decision-making is the lack of tractable concentration inequalities required for principled exploration. We overcome this limitation by directly modeling the learning curve behavior of the underlying estimator. Specifically, assuming the Mean Squared Error decreases as a power law in the number of training samples, we derive a generalized concentration inequality and prove that ML-UCB achieves sublinear regret. This framework enables the principled integration of any ML model whose learning curve can be empirically characterized, eliminating the need for model-specific theoretical analysis. We validate our approach through experiments on a collaborative filtering recommendation system using online matrix factorization with synthetic data designed to simulate a simplified two-tower model, demonstrating substantial improvements over LinUCB
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。