用扩散模型做上下文老虎机的后验采样，兼顾效率与复杂分布建模。
Online Posterior Sampling with a Diffusion Prior
- 通过反向扩散过程分阶段采样近似后验，每阶段用拉普拉斯近似。
- 在多种上下文老虎机任务中表现良好，且渐近一致。
- 适合需要复杂先验建模的强化学习场景，如推荐系统。
在上下文老虎机问题中，使用高斯先验可精确或近似实现后验采样，但难以刻画复杂分布。本文提出基于扩散模型先验的近似后验采样算法，核心思想是沿反向扩散过程分阶段采样近似条件后验，各阶段通过拉普拉斯近似获得。该方法借鉴高斯先验下的后验采样思路，保持简洁高效，并具有渐近一致性。实验表明其在多种上下文老虎机任务中表现优异。
原文摘要 · Abstract (English)
Posterior sampling in contextual bandits with a Gaussian prior can be implemented exactly or approximately using the Laplace approximation. The Gaussian prior is computationally efficient but it cannot describe complex distributions. In this work, we propose approximate posterior sampling algorithms for contextual bandits with a diffusion model prior. The key idea is to sample from a chain of approximate conditional posteriors, one for each stage of the reverse diffusion process, which are obtained by the Laplace approximation. Our approximations are motivated by posterior sampling with a Gaussian prior, and inherit its simplicity and efficiency. They are asymptotically consistent and perform well empirically on a variety of contextual bandit problems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。