用随机函数与仿射基准混合训练神经网络,自动配置优化算法性能更优。
Landscape-Aware Automated Algorithm Configuration using Multi-output Mixed Regression and Classification
- 用随机生成函数和仿射基准混合数据训练神经网络,处理多输出回归与分类任务。
- 在5维和20维问题上,所提方法能识别接近最优的算法配置。
- 适合缺乏自动配置经验的研究者,尤其在优化算法调参场景中表现突出。
在景观感知的算法选择问题中,基于特征的预测模型效果高度依赖于训练数据的代表性。本文研究了随机生成函数(RGF)在模型训练中的潜力,其覆盖的优化问题类别远比广泛使用的黑箱优化基准(BBOB)套件更丰富。聚焦于自动算法配置(AAC),即根据问题实例的景观特征选择最优算法并调优超参数。我们分析了全连接神经网络在不同训练数据集(如RGF和多仿射BBOB,MA-BBOB)下的多输出混合回归与分类表现。基于5维和20维BBOB函数的实验结果表明,所提方法可识别近似最优配置,大多数情况下优于实践中常见的默认配置。此外,预测配置在许多情况下可媲美单一最优求解器。总体而言,结合RGF与MA-BBOB训练的神经网络模型能最好地识别高性能配置。
原文摘要 · Abstract (English)
In landscape-aware algorithm selection problem, the effectiveness of feature-based predictive models strongly depends on the representativeness of training data for practical applications. In this work, we investigate the potential of randomly generated functions (RGF) for the model training, which cover a much more diverse set of optimization problem classes compared to the widely-used black-box optimization benchmarking (BBOB) suite. Correspondingly, we focus on automated algorithm configuration (AAC), that is, selecting the best suited algorithm and fine-tuning its hyperparameters based on the landscape features of problem instances. Precisely, we analyze the performance of dense neural network (NN) models in handling the multi-output mixed regression and classification tasks using different training data sets, such as RGF and many-affine BBOB (MA-BBOB) functions. Based on our results on the BBOB functions in 5d and 20d, near optimal configurations can be identified using the proposed approach, which can most of the time outperform the off-the-shelf default configuration considered by practitioners with limited knowledge about AAC. Furthermore, the predicted configurations are competitive against the single best solver in many cases. Overall, configurations with better performance can be best identified by using NN models trained on a combination of RGF and MA-BBOB functions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。