用历史经验动态缩小搜索空间，让AutoML更快更准。
Dynamic Design of Machine Learning Pipelines via Metalearning
- 基于元知识筛选高潜力搜索区域，减少无效探索。
- 随机搜索加速89%，搜索空间缩小1.8~4.3倍，性能基本不变。
- 适合追求效率的AutoML开发者，尤其擅长降低计算开销。
自动化机器学习（AutoML）通过自动完成模型选择、超参数调优和特征工程，降低了机器学习系统的使用门槛。然而，传统搜索与优化策略（如随机搜索、粒子群优化、贝叶斯优化）带来的高计算成本仍是主要挑战。此外，AutoML系统通常在庞大的搜索空间中探索，易导致过拟合。本文提出一种基于元学习的动态搜索空间设计方法，利用历史元知识识别有潜力的搜索区域，加速优化过程。实验表明，该方法在随机搜索中可将运行时间减少89%，搜索空间分别缩小1.8/13（预处理器）和4.3/16（分类器），且预测性能未显著下降。该方法在Auto-Sklearn上的应用也证明其有效。研究还深入探讨了元特征选择、元模型可解释性及搜索空间压缩策略的权衡问题。
原文摘要 · Abstract (English)
Automated machine learning (AutoML) has democratized the design of machine learning based systems, by automating model selection, hyperparameter tuning and feature engineering. However, the high computational cost associated with traditional search and optimization strategies, such as Random Search, Particle Swarm Optimization and Bayesian Optimization, remains a significant challenge. Moreover, AutoML systems typically explore a large search space, which can lead to overfitting. This paper introduces a metalearning method for dynamically designing search spaces for AutoML system. The proposed method uses historical metaknowledge to select promising regions of the search space, accelerating the optimization process. According to experiments conducted for this study, the proposed method can reduce runtime by 89\% in Random Search and search space by (1.8/13 preprocessor and 4.3/16 classifier), without compromising significant predictive performance. Moreover, the proposed method showed competitive performance when adapted to Auto-Sklearn, reducing its search space. Furthermore, this study encompasses insights into meta-feature selection, meta-model explainability, and the trade-offs inherent in search space reduction strategies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。