用微调大模型实现高效贝叶斯优化，省去复杂采样计算。
Thompson Sampling via Fine-Tuning of LLMs
- 通过微调语言模型直接建模候选最优概率，避免最大化采集函数。
- 在三个任务中均达到顶尖样本效率与计算效率。
- 适合需要快速搜索高维离散空间的科研与工程场景。
在大规模非结构化离散空间中的贝叶斯优化常因缺乏梯度而难以最大化采集函数，导致计算成本高昂。本文提出一种可扩展的汤普森采样方法——基于微调的大语言模型汤普森采样（ToSFiT），通过直接参数化候选项获得最大奖励的概率，消除对采集函数最大化的依赖。该方法利用提示条件下的大语言模型所蕴含的先验知识，并逐步适配至后验分布。理论上，我们为变分形式的汤普森采样推导出新的遗憾界，其性能与标准版本相当。分析表明，对最大可能性后验概率的精准适应至关重要，这一原则构成了ToSFiT算法的核心。实验上，在问答响应优化、热稳定蛋白质搜索和量子电路设计三个任务中，与基于上下文的贝叶斯优化、强化学习及进化搜索等方法相比，ToSFiT展现出卓越的样本效率和计算效率。
原文摘要 · Abstract (English)
Bayesian optimization in large unstructured discrete spaces is often hindered by the computational cost of maximizing acquisition functions due to the absence of gradients. We propose a scalable alternative based on Thompson sampling that eliminates the need for acquisition function maximization by directly parameterizing the probability that a candidate yields the maximum reward. Our approach, Thompson Sampling via Fine-Tuning (ToSFiT) leverages the prior knowledge embedded in prompt-conditioned large language models, and incrementally adapts them toward the posterior. Theoretically, we derive a novel regret bound for a variational formulation of Thompson Sampling that matches the strong guarantees of its standard counterpart. Our analysis reveals the critical role of careful adaptation to the posterior probability of maximality -- a principle that underpins our ToSFiT algorithm. Empirically, we validate our method on three diverse tasks: FAQ response refinement, thermally stable protein search, and quantum circuit design. Within a collection of methods covering in-context Bayesian optimization, reinforcement learning, and evolutionary search, ToSFiT exhibits both state-of-the-art sample efficiency and computational efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。