arXiv:2511.02570cs.LG2025-11被引 2

让用户持续影响超参优化,动态调整先验提升效率。

Dynamic Priors in Bayesian Optimization for Hyperparameter Optimization

  • 引入动态先验机制,随时间衰减并加权用户输入。
  • 在多种基准上优于现有方法,无论先验是否准确。
  • 适合需要人机协作的迭代式模型开发场景。

贝叶斯优化(BO)是超参数优化(HPO)的常用方法。然而,现有方法仅在初始化时融入专家知识,限制了实践者在新洞察出现后对优化过程的影响,制约了其在迭代式机器学习开发流程中的应用。我们提出DynaBO,一种支持持续用户控制的BO框架。DynaBO通过将随时间衰减的、加权的先验偏好加入采集函数,逐步利用用户提供的先验,同时保持渐近收敛性保证。为增强鲁棒性,引入基于代理模型的防护机制,可检测并可能拒绝误导性先验。理论证明显示:近必然收敛、对欺骗性先验具鲁棒性、在提供有效先验时加速收敛。在多个HPO基准上的大量实验表明,DynaBO在所有基准和所有类型的先验下均持续优于现有最优方法。结果表明,DynaBO实现了可靠高效的协同贝叶斯优化,连接自动化与人工控制的模型开发流程。

原文摘要 · Abstract (English)

Bayesian optimization (BO) is a widely used approach to hyperparameter optimization (HPO). However, most existing HPO methods only incorporate expert knowledge during initialization, limiting practitioners' ability to influence the optimization process as new insights emerge. This limits the applicability of BO in iterative machine learning development workflows. We propose DynaBO, a BO framework that enables continuous user control of the optimization process. Over time, DynaBO leverages provided user priors by augmenting the acquisition function with decaying, prior-weighted preferences while preserving asymptotic convergence guarantees. To enhance robustness, we introduce a surrogate-model-based safeguard that detects and, possibly, rejects misleading priors. We prove theoretical results on near-certain convergence, robustness to deceptive priors, and accelerated convergence when informative priors are provided. Extensive experiments across various HPO benchmarks show that DynaBO consistently outperforms state-of-the-art competitors across all benchmarks and for all prior kinds. Our results demonstrate that DynaBO enables reliable and efficient collaborative BO, bridging automated and manually controlled model development.

超参优化贝叶斯优化人机协同

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。