arXiv:2605.10137stat.MLcs.LG2026-05被引 1

用神经网络实现更精准的上下文老虎机决策,提升推荐与医疗等场景效果。

PFN-TS: Thompson Sampling for Contextual Bandits via Prior-Data Fitted Networks

论文配图:PFN-TS: Thompson Sampling for Contextual Bandits via Prior-Data Fitted Networks
图 1 · 摘自论文原文
  • 通过子采样中心极限定理将神经网络预测转为奖励均值样本
  • 仅需 $O(\log n)$ 数据前缀估计方差,计算效率远超以往方法
  • 在合成数据和真实医疗数据上表现最佳,适合高维决策场景

Thompson采样是上下文老虎机中广泛使用的方法:每轮从贝叶斯后验中采样一个奖励函数,并据此贪婪行动。先验-数据拟合网络(PFNs)如TabPFN v2+和TabICL v2因其能在单次前向传播中逼近贝叶斯后验预测分布,成为理想候选。然而,PFNs预测的未来奖励存在噪声,而Thompson采样需要对潜在均值奖励函数的不确定性建模。本文提出PFN-TS,一种将PFN后验预测转化为均值奖励样本的采样算法,利用子采样预测中心极限定理实现转换。该方法仅需$O(\log n)$个数据前缀的几何网格来估计后验方差,而非以往方法所需的$O(n)$完整预测序列,并复用TabICL的缓存表示以跨轮次共享。我们证明了子采样方差估计器的一致性,并给出了贝叶斯后悔界,其将PFN-TS的后悔分解为基于PFN先验的精确后验采样后悔加上近似项。实验表明,PFN-TS在非线性合成数据和OpenML分类转老虎机基准中平均排名最高,在线性及BART生成奖励上仍具竞争力,并在离线移动健康评估中取得最高估计策略价值。代码已公开于https://anonymous.4open.science/r/PFN_TS-36ED/。

原文摘要 · Abstract (English)

Thompson sampling is a widely used strategy for contextual bandits: at each round, it samples a reward function from a Bayesian posterior and acts greedily under that sample. Prior-data fitted networks (PFNs), such as TabPFN v2+ and TabICL v2, are attractive candidates for this purpose because they approximate Bayesian posterior predictive distributions in a single forward pass. However, PFNs predict noisy future rewards, while Thompson sampling requires uncertainty over the latent mean reward function. We propose PFN-TS, a Thompson sampling algorithm that converts PFN posterior predictives into mean-reward samples using a subsampled predictive central limit theorem. The method estimates posterior variance from a geometric grid of $O(\log n)$ dataset prefixes rather than the full $O(n)$ predictive sequence used in previous predictive-sequence approaches, and reuses TabICL's cached representations across rounds. We prove consistency of the subsampled variance estimator and give a Bayesian regret bound that decomposes PFN-TS regret into exact posterior-sampling regret under the PFN prior plus approximation terms. Empirically, PFN-TS achieves the best average rank across nonlinear synthetic and OpenML classification-to-bandit benchmarks, remains competitive on linear and BART-generated rewards, and attains the highest estimated policy value in an offline mobile-health evaluation. Code is available at https://anonymous.4open.science/r/PFN_TS-36ED/.

上下文老虎机贝叶斯优化神经网络

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。