比较回归与成对模型在启发式优化中的表现,发现成对模型更适配某些算法。
Comparative study of regression vs pairwise models for surrogate-based heuristic optimisation
- 用回归和成对分类两种方式构建代理模型,前者预测具体适应度值,后者判断解优劣关系。
- 在连续优化与工业挑战问题上,成对模型在多数配置下表现更优,尤其适合无需精确值的算法。
- 适合研究代理模型与进化算法结合的学者,尤其关注计算效率与模型使用方式的场景。
启发式优化算法通过采样解、评估其适应度并引导搜索向有潜力的方向进行。但在许多情况下,适应度函数涉及昂贵的计算,严重限制了可执行评估的数量。在此背景下,代理模型成为缓解计算负担的优秀替代方案。本文将代理问题建模为两类方法:一类是逼近适应度值的回归模型(表面代理模型),另一类是新颖的分类模型(成对代理模型)。成对方法可直接被某些算法(如差分进化)利用,因其仅需知道一个解是否优于另一个,而无需具体适应度值。基于这两种建模方式,本文在不同配置下进行了多维度分析:采用多种机器学习算法(正则化回归、神经网络、决策树、提升方法、随机森林)、不同的代理策略(鼓励多样性或放松预测阈值),并对比表面与成对代理模型。实验部分涵盖SOCO2011竞赛提出的连续优化基准问题及GECCO2021工业挑战中的仿真问题。结果表明,在使用在线机器学习代理模型时,整体搜索性能不仅取决于预测模型的准确性,还受对正负样本偏差类型以及优化算法如何利用预测结果决定是否执行真实适应度函数的影响。
原文摘要 · Abstract (English)
Heuristic optimisation algorithms explore the search space by sampling solutions, evaluating their fitness, and biasing the search in the direction of promising solutions. However, in many cases, this fitness function involves executing expensive computational calculations, drastically reducing the reasonable number of evaluations. In this context, surrogate models have emerged as an excellent alternative to alleviate these computational problems. This paper addresses the formulation of surrogate problems as both regression models that approximate fitness (surface surrogate models) and a novel way to connect classification models (pairwise surrogate models). The pairwise approach can be directly exploited by some algorithms, such as Differential Evolution, in which the fitness value is not actually needed to drive the search, and it is sufficient to know whether a solution is better than another one or not. Based on these modelling approaches, we have conducted a multidimensional analysis of surrogate models under different configurations: different machine learning algorithms (regularised regression, neural networks, decision trees, boosting methods, and random forests), different surrogate strategies (encouraging diversity or relaxing prediction thresholds), and compare them for both surface and pairwise surrogate models. The experimental part of the article includes the benchmark problems already proposed for the SOCO2011 competition in continuous optimisation and a simulation problem included in the recent GECCO2021 Industrial Challenge. This paper shows that the performance of the overall search, when using online machine learning-based surrogate models, depends not only on the accuracy of the predictive model but also on both the kind of bias towards positive or negative cases and how the optimisation uses those predictions to decide whether to execute the actual fitness function.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。