针对上尾性能的强化学习，提出新型自适应置信量化算法
Conformal-Style Quantile Analyses for Stochastic Bandits
- 用自适应置信估计结合乐观奖励，优化上尾表现
- 理论证明其对上尾损失的累积误差为对数级增长
- 适合关注极端收益而非平均收益的决策场景
传统随机老虎机算法通常基于均值收益进行分析，但许多实际问题更关注上尾表现优异的选项。本文设定固定误覆盖率α，以第1−α/2分位数作为臂j的上尾目标值。该目标可能与均值排序不同，造成与经典带宽目标的偏差。为此，提出ACP-UCB1算法，融合自适应置信估计与UCB型乐观激励。技术难点在于：符合性得分依赖于动态更新的经验分位数估计，并在自适应水平下评估。通过奖励-分位数集中性、评分分位数的扰动分析及自适应水平的确定性定位，控制该端点。理论证明ACP-UCB1实现对数上尾后悔，每臂贡献为O(log n / Δ_j^ACP)。还提供与UCB1的指标相关后悔分解，并通过数值实验验证性能提升。
原文摘要 · Abstract (English)
Stochastic bandit algorithms are usually analyzed under a mean-reward criterion, yet many problems favor arms with strong upper-tail performance, which we study herein. For a fixed miscoverage level \(α\), the natural upper-tail target of arm \(j\) is the upper endpoint \(F_j^{-1}(1-α/2)\) of a central prediction interval. This target can rank arms differently from their means, creating a central mismatch with the classical bandit objective. To this end, we propose ACP-UCB1, a conformal-style policy that combines an adaptive conformal estimate of the upper endpoint with a UCB-type optimism bonus. The technical challenge is that the conformity scores used by ACP-UCB1 are recomputed from evolving empirical quantile estimates and evaluated at an adaptive level. We control this endpoint through reward-quantile concentration, a perturbation argument for recomputed score quantiles, and deterministic localization of the adaptive level. ACP-UCB1 achieves logarithmic upper-quantile regret with per-arm contribution \(O(\nicefrac{\log n}{Δ_j^{\mathrm{ACP}}})\). We also provide metric-specific regret decompositions comparing ACP-UCB1 with UCB1 and use numerical experiments to validate performance and improvement.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。