用贝叶斯树模型提升健康干预中的个性化决策效果
BFTS: Thompson Sampling with Bayesian Additive Regression Trees
- 将贝叶斯加性回归树直接融入探索过程,实现概率化决策
- 在表格数据上达到领先表现,不确定性估计接近理想水平
- 适合需要高可靠性在线决策的个性化医疗应用
上下文相关老虎机是个性化移动健康干预的核心技术,需适应复杂的非线性用户行为。尽管汤普森采样(TS)是首选策略,其性能依赖于奖励模型质量。传统线性模型偏差高,神经网络在在线设置中常不稳定且难调参。树集成虽在表格数据上占优,但通常依赖启发式不确定性估计,缺乏为TS提供理论支撑的概率基础。本文提出贝叶斯森林汤普森采样(BFTS),首个将贝叶斯加性回归树(BART)——一种全概率化的树之和模型——直接整合进探索循环的上下文相关老虎机算法。我们证明了BFTS在理论上是合理的,推导出信息论意义下的贝叶斯后悔界为$ ilde{O}( ext{sqrt}{T})$。作为补充结果,我们还建立了‘感觉良好’变体的频率学极小极大最优性,证实了BART先验对非参数老虎机的结构适配性。实证表明,BFTS在表格基准测试中达到最先进后悔率,且不确定性校准接近名义水平。此外,在‘少喝酒’微随机试验的离线策略评估中,相比部署策略,BFTS使参与率提升超30%,展示了其在行为干预中的实际有效性。
原文摘要 · Abstract (English)
Contextual bandits are a core technology for personalized mobile health interventions, where decision-making requires adapting to complex, non-linear user behaviors. While Thompson Sampling (TS) is a preferred strategy for these problems, its performance hinges on the quality of the underlying reward model. Standard linear models suffer from high bias, while neural network approaches are often brittle and difficult to tune in online settings. Conversely, tree ensembles dominate tabular data prediction but typically rely on heuristic uncertainty quantification, lacking a principled probabilistic basis for TS. We propose Bayesian Forest Thompson Sampling (BFTS), the first contextual bandit algorithm to integrate Bayesian Additive Regression Trees (BART), a fully probabilistic sum-of-trees model, directly into the exploration loop. We prove that BFTS is theoretically sound, deriving an information-theoretic Bayesian regret bound of $\tilde{O}(\sqrt{T})$. As a complementary result, we establish frequentist minimax optimality for a "feel-good" variant, confirming the structural suitability of BART priors for non-parametric bandits. Empirically, BFTS achieves state-of-the-art regret on tabular benchmarks with near-nominal uncertainty calibration. Furthermore, in an offline policy evaluation on the Drink Less micro-randomized trial, BFTS improves engagement rates by over 30% compared to the deployed policy, demonstrating its practical effectiveness for behavioral interventions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。