用自适应筛选加速自动化机器学习,提升效率与效果
BOASF: A Unified Framework for Speeding up Automatic Machine Learning via Adaptive Successive Filtering
- 结合贝叶斯优化与自适应筛选,在多臂老虎机框架下智能选择配置
- 相比现有方法,显著缩短搜索时间,且在不同预算下表现更优
- 适合资源有限但追求高效模型调优的实践者使用
机器学习在众多领域取得成功,但对非专家而言,高效完成任务仍具挑战。从大量可能的模型或超参数组合中寻找最优解,通常需要大量专业知识。为此,我们提出一种统一的多臂老虎机框架下的联合贝叶斯优化与自适应连续筛选算法(BOASF),用于自动化模型选择或超参数优化。具体而言,BOASF 在每轮评估中利用贝叶斯优化为每个臂选择有潜力的配置;随后,基于高斯上置信界(Gaussian UCB)的概率模型,自适应地提前淘汰表现差的臂;同时采用Softmax模型动态分配资源,使更有希望推进的臂获得更多计算资源。实验表明,相较于现有最先进的自动化机器学习方法,BOASF在加速模型选择与超参数优化的同时,实现了更稳健且更优的预测性能。此外,在不同时间预算下,BOASF表现出更佳的任意时间性能。
原文摘要 · Abstract (English)
Machine learning has been making great success in many application areas. However, for the non-expert practitioners, it is always very challenging to address a machine learning task successfully and efficiently. Finding the optimal machine learning model or the hyperparameter combination set from a large number of possible alternatives usually requires considerable expert knowledge and experience. To tackle this problem, we propose a combined Bayesian Optimization and Adaptive Successive Filtering algorithm (BOASF) under a unified multi-armed bandit framework to automate the model selection or the hyperparameter optimization. Specifically, BOASF consists of multiple evaluation rounds in each of which we select promising configurations for each arm using the Bayesian optimization. Then, ASF can early discard the poor-performed arms adaptively using a Gaussian UCB-based probabilistic model. Furthermore, a Softmax model is employed to adaptively allocate available resources for each promising arm that advances to the next round. The arm with a higher probability of advancing will be allocated more resources. Experimental results show that BOASF is effective for speeding up the model selection and hyperparameter optimization processes while achieving robust and better prediction performance than the existing state-of-the-art automatic machine learning methods. Moreover, BOASF achieves better anytime performance under various time budgets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。