用非马尔可夫强化学习解决多目标贝叶斯优化难题
BOFormer: Learning to Solve Multi-Objective Bayesian Optimization via Non-Markovian RL
- 引入序列建模的BOFormer框架，突破多目标优化的非马尔可夫性限制
- 在合成与真实场景中持续优于基准算法，提升超参数优化效率
- 适合研究多目标优化、强化学习与自动机器学习的学者参考
贝叶斯优化（BO）通过高斯过程先验和采集函数（AF）高效优化黑箱函数。近期基于学习的采集函数在单目标优化中表现优异，因其具备非贪婪特性。然而，将此类方法直接扩展至多目标贝叶斯优化（MOBO）时，会遭遇“超体积可辨识性问题”，源于MOBO问题的非马尔可夫性质。受非马尔可夫强化学习文献及Transformer在语言建模中的成功启发，本文提出一种广义深度Q学习框架，并构建了基于序列建模的BOFormer，以应对该挑战。通过大量实验验证，BOFormer在多种合成和真实世界的多目标超参数优化任务中，始终优于规则基与学习基的基准算法。源代码已公开，以促进该方向的进一步研究。
原文摘要 · Abstract (English)
Bayesian optimization (BO) offers an efficient pipeline for optimizing black-box functions with the help of a Gaussian process prior and an acquisition function (AF). Recently, in the context of single-objective BO, learning-based AFs witnessed promising empirical results given its favorable non-myopic nature. Despite this, the direct extension of these approaches to multi-objective Bayesian optimization (MOBO) suffer from the \textit{hypervolume identifiability issue}, which results from the non-Markovian nature of MOBO problems. To tackle this, inspired by the non-Markovian RL literature and the success of Transformers in language modeling, we present a generalized deep Q-learning framework and propose \textit{BOFormer}, which substantiates this framework for MOBO via sequence modeling. Through extensive evaluation, we demonstrate that BOFormer constantly outperforms the benchmark rule-based and learning-based algorithms in various synthetic MOBO and real-world multi-objective hyperparameter optimization problems. We have made the source code publicly available to encourage further research in this direction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。