arXiv:2503.16941stat.MLcs.LG2025-03

针对高维特征的在线决策,提出无需参数假设的新算法。

SPARKLE: A Nonparametric Approach for Online Decision-Making with High-Dimensional Covariates

  • 用双重惩罚估计器建模复杂奖励与特征关系
  • 理论证明后悔值随维度对数增长,首次实现此结果
  • 适合推荐系统、精准医疗等高维数据场景

个性化服务是当今数字经济的核心,其序列决策常被建模为上下文相关多臂老虎机问题。现代应用面临两大挑战:高维协变量和需要非参数模型以捕捉复杂的奖励-协变量关系。我们提出 SPARKLE,一种基于稀疏加性奖励模型的新型上下文相关多臂老虎机算法,通过(i)双重惩罚估计器进行非参数奖励估计,以及(ii)基于轮次的设计与自适应筛选机制,平衡探索与利用。我们证明了后悔值呈次线性增长,且仅随协变量维度对数增长;据我们所知,这是首个针对高维协变量的非参数上下文相关多臂老虎机的此类结果。我们还推导了信息论下界,当奖励平滑度增加时,上界与下界差距趋于消失。在合成数据及视频推荐、个性化医学真实数据上的大量实验表明,该方法在高维场景下表现优异。

原文摘要 · Abstract (English)

Personalized services are central to today's digital economy, and their sequential decisions are often modeled as contextual bandits. Modern applications pose two main challenges: high-dimensional covariates and the need for nonparametric models to capture complex reward-covariate relationships. We propose SPARKLE, a novel contextual bandit algorithm based on a sparse additive reward model that addresses both challenges through (i) a doubly penalized estimator for nonparametric reward estimation and (ii) an epoch-based design with adaptive screening to balance exploration and exploitation. We prove a sublinear regret bound that grows only logarithmically in the covariate dimensionality; to our knowledge, this is the first such result for nonparametric contextual bandits with high-dimensional covariates. We also derive an information-theoretic lower bound, and the gap to the upper bound vanishes as the reward smoothness increases. Extensive experiments on synthetic data and real data from video recommendation and personalized medicine show strong performance in high-dimensional settings.

在线决策高维数据非参数模型上下文多臂

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。