arXiv:2601.21167cs.LG2026-01被引 1

提出两种自适应曲率的推荐算法,显著提升低回报但信息量大的动作利用效率。

Learning What to Recommend: Minimax Optimal Simple Regret in Logistic Bandits

  • 基于动作特征曲率设计探索策略,动态权衡奖励与信息价值
  • 理论证明最优简单后悔上界为 $\tilde O(d/\sqrt{κ_* T})$,与下界匹配
  • 适用于需精准推荐且特征维度高的场景,如个性化推荐系统

研究了在 $d$ 维动作特征下的随机逻辑回归强化学习问题,目标是最小化简单后悔。由于逻辑函数的局部曲率影响信息量,高即时奖励的动作未必最利于发现最优推荐。本文揭示,最小最大难度由最优动作处的 sigmoid 斜率倒数 $κ_*$ 决定。通过构造饱和硬族实例,展示了信息受限与推荐错误损失并存的难题机制。提出了两个曲率感知算法:– MULog,一种纯探索方法,其最终推荐的高概率上界为 $\tilde O(d/\sqrt{κ_* T})$,与下界对齐;– THATS,一种类汤普森采样方法,计算更轻量。实验验证:低回报但信息丰富的动作可显著降低任务难度,且曲率感知方法表现更优。

原文摘要 · Abstract (English)

We study stochastic logistic bandits with $d$-dimensional action features under the simple-regret objective, where a learner uses $T$ rounds of exploration to output a single final action. The logistic structure is essential here: because the informativeness of an action depends on the local curvature of the sigmoid, actions that are best for immediate reward need not be the most useful for identifying the best final recommendation. We show that the first-order minimax difficulty is governed by $κ_*$, the inverse slope of the sigmoid at the optimal action. The lower bound is realized by a shifted saturated hard family in which saturation simultaneously limits the information available about the final decision and controls the value loss from a wrong recommendation. This reveals a hard mechanism distinct from cumulative-regret constructions, even though online-to-batch reductions recover the same leading order in expectation. We then develop two curvature-aware algorithms: \MULog, a pure-exploration method whose final recommendation satisfies a high-probability upper bound of order $\tilde O(d/\sqrt{κ_* T})$, matching the lower bound up to logarithmic factors, and \THATS, a Thompson-sampling-style method that provides a computationally lighter alternative. Experiments on both hard and easy geometries support the same picture: informative low-reward actions can make instances substantially easier, and the curvature-aware methods exploit this structure especially effectively.

强化学习推荐系统在线学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。