用序贯蒙特卡洛方法高效优化无法计算梯度的函数
Efficient Stochastic Optimisation via Sequential Monte Carlo
- 用SMC采样替代传统耗时的内层采样，提升优化效率
- 理论证明了算法基本递推关系的收敛性
- 在能量模型奖励调优中验证了显著计算优势
在机器学习与统计中，许多问题涉及无法直接计算梯度的目标函数，如最大边际似然估计和生成模型微调。传统随机逼近方法需通过内层采样循环获得有偏的随机梯度估计，计算成本高。本文提出基于序贯蒙特卡洛（SMC）的优化方法，以高效SMC近似替代昂贵的内层采样，实现显著计算增益。我们建立了该方法基本递推关系的收敛性理论，并在多种设置下验证了其在能量基模型奖励调优中的有效性。
原文摘要 · Abstract (English)
The problem of optimising functions with intractable gradients frequently arises in machine learning and statistics, ranging from maximum marginal likelihood estimation procedures to fine-tuning of generative models. Stochastic approximation methods for this class of problems typically require inner sampling loops to obtain (biased) stochastic gradient estimates, which rapidly becomes computationally expensive. In this work, we develop sequential Monte Carlo (SMC) samplers for optimisation of functions with intractable gradients. Our approach replaces expensive inner sampling methods with efficient SMC approximations, which can result in significant computational gains. We establish convergence results for the basic recursions defined by our methodology which SMC samplers approximate. We demonstrate the effectiveness of our approach on the reward-tuning of energy-based models within various settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。