优化后置堆叠集成，提升自动机器学习预测效果
PSEO: Optimizing Post-hoc Stacking Ensemble Through Hyperparameter Tuning
- 用二元二次规划选基模型，平衡多样性与性能
- 在80个数据集上平均排名第二，优于16种方法
- 适合需要高精度集成预测的AutoML研究者
组合算法选择与超参数优化（CASH）是自动化机器学习（AutoML）的核心问题。受集成学习成功的启发，近年的AutoML系统通过构建后置集成进行最终预测，而非依赖单一最优模型。然而，多数CASH方法虽在单模型搜索上投入大量计算，但在集成阶段仍采用固定策略，无法根据任务特性自适应调整。为此，我们提出PSEO框架，用于优化后置堆叠集成。首先，通过二元二次规划进行基模型选择，权衡多样性和性能；其次，引入两种机制充分挖掘多层堆叠潜力；最后，构建超参数空间并搜索最优后置集成策略。在80个公开数据集上的实验表明，该方法在16种对比方法中取得最佳平均测试排名（2.96），涵盖近期AutoML系统的后置设计及前沿集成学习方法。
原文摘要 · Abstract (English)
The Combined Algorithm Selection and Hyperparameter Optimization (CASH) problem is fundamental in Automated Machine Learning (AutoML). Inspired by the success of ensemble learning, recent AutoML systems construct post-hoc ensembles for final predictions rather than relying on the best single model. However, while most CASH methods conduct extensive searches for the optimal single model, they typically employ fixed strategies during the ensemble phase that fail to adapt to specific task characteristics. To tackle this issue, we propose PSEO, a framework for post-hoc stacking ensemble optimization. First, we conduct base model selection through binary quadratic programming, with a trade-off between diversity and performance. Furthermore, we introduce two mechanisms to fully realize the potential of multi-layer stacking. Finally, PSEO builds a hyperparameter space and searches for the optimal post-hoc ensemble strategy within it. Empirical results on 80 public datasets show that \sys achieves the best average test rank (2.96) among 16 methods, including post-hoc designs in recent AutoML systems and state-of-the-art ensemble learning methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。