arXiv:2502.09329cs.LG2025-02KDD被引 1

用共享隐空间加速算法与超参联合搜索,减少试错次数。

Bayesian Optimization for Simultaneous Selection of Machine Learning Algorithms and Hyperparameters on Shared Latent Space

  • 将不同算法的超参空间映射到统一隐空间,实现信息共享。
  • 在OpenML数据集上仅需少量观测即达到最优性能。
  • 适合需要快速调优多种算法的机器学习工程场景。

选择最佳的机器学习算法及其超参数组合对构建高性能系统至关重要。然而,由于候选算法与超参数组合数量庞大,穷举验证耗时巨大。现有研究多采用贝叶斯优化(BO)加速搜索,但通常面临各算法超参空间差异大、需独立建模的问题,导致每个算法都需要大量观测。本文提出将不同算法的超参空间嵌入共享隐空间,构建多任务代理模型进行贝叶斯优化,通过共享观测信息提升效率。进一步引入对抗正则化预训练隐空间,并设计排序模型以选择适配目标数据集的预训练嵌入。实证研究在OpenML数据集上验证了方法的有效性。

原文摘要 · Abstract (English)

Selecting the optimal combination of a machine learning (ML) algorithm and its hyper-parameters is crucial for the development of high-performance ML systems. However, since the combination of ML algorithms and hyper-parameters is enormous, the exhaustive validation requires a significant amount of time. Many existing studies use Bayesian optimization (BO) for accelerating the search. On the other hand, a significant difficulty is that, in general, there exists a different hyper-parameter space for each one of candidate ML algorithms. BO-based approaches typically build a surrogate model independently for each hyper-parameter space, by which sufficient observations are required for all candidate ML algorithms. In this study, our proposed method embeds different hyper-parameter spaces into a shared latent space, in which a surrogate multi-task model for BO is estimated. This approach can share information of observations from different ML algorithms by which efficient optimization is expected with a smaller number of total observations. We further propose the pre-training of the latent space embedding with an adversarial regularization, and a ranking model for selecting an effective pre-trained embedding for a given target dataset. Our empirical study demonstrates effectiveness of the proposed method through datasets from OpenML.

贝叶斯优化超参搜索多任务学习算法选择

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。