arXiv:2607.06879cs.LGstat.ML2026-07被引 1

用廉价代理信号提升最优选项识别效率,理论证明可大幅减少实验次数。

Best-Arm Identification with Generative Proxy

论文配图:Best-Arm Identification with Generative Proxy
图 1 · 摘自论文原文
  • 通过代理变量与真实奖励的协方差校正,构建异方差识别模型。
  • 在相关性未知时仍能实现接近已知相关性的最优样本复杂度。
  • 适用于高成本决策场景,如金融定价、推荐系统等应用。

最优臂识别是数据驱动决策的经典模型,但许多应用中每次奖励观测代价高昂。本文研究固定置信度下的最优臂识别问题,其中每个昂贵的奖励抽取都配有一个廉价但相关的代理评分。代理的均值可在离线阶段估计并视为已知,而其与奖励的相关系数ρ(决定代理帮助程度)未知,需在线学习。我们发现,控制变量调整将此模型转化为异方差识别问题,其最优样本复杂度可改善为残差方差1−ρ²。核心难点在于相关系数必须从同一昂贵样本中学习,且直接使用残差方差估计会低估风险,导致方法失效。为此,提出PROBE(PRoxy OLS for Best-arm Exploration),一种相位消除算法,通过普通最小二乘法直接维护残差方差的上界,并利用精确的卡方分布确保证书在任意未知相关性下仍有效。理论上证明PROBE是δ-PAC,且达到已知相关性下的最优样本复杂度,仅相差常数倍乘性和常数加性校准成本。该保证在(ε,δ)-PAC设置下亦成立。合成实例及基于大语言模型和表格代理的自动贷款定价回放实验表明,PROBE的样本节省程度与奖励-代理相关性强度完全匹配理论预测。

原文摘要 · Abstract (English)

Best-arm identification is a canonical model for data-driven decision-making, but in many applications each reward observation is costly. Motivated by the growing availability of cheap predictions from machine learning and large language models, we study fixed-confidence best-arm identification in which each costly reward pull is paired with a cheap but correlated proxy score. The marginal mean of the proxy can be estimated offline and is treated as known, whereas its correlation $ρ$ with the reward, which governs how much the proxy helps, is unknown and must be learned online in pair with real rewards. We show that a control-variate adjustment turns this model into a heteroscedastic identification problem whose oracle sample complexity improves by residual variance $1-ρ^2$. The central difficulty is that the correlation must be learned from the same costly samples that identification consumes online, and that a plug-in estimate of the residual variance is anti-conservative and can compromise correctness. We propose PROBE (PRoxy OLS for Best-arm Exploration), a phase-elimination algorithm that directly maintains an upper certificate on the residual variance with an ordinary least squares fit, whose exact chi-square law keeps the certificate valid regardless of the unknown correlation. We prove that PROBE is $δ$-PAC and attains the known-correlation oracle sample complexity up to a constant multiplicative factor and a constant additive calibration cost. The guarantee extends to the $(ε,δ)$-PAC setting under minimal changes to the algorithm. Numerical experiments on synthetic instances and on an auto-loan pricing replay with large language model and tabular proxies confirm that the sample savings of PROBE scale with the strength of the reward-proxy correlation, exactly as the theory predicts.

最优臂识别代理信号样本效率强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。