arXiv:2501.01905cs.LG2025-01被引 9

用多目标优化解决符号回归过拟合问题,提升小数据集表现。

Alleviating Overfitting in Transformation-Interaction-Rational Symbolic Regression with Multi-Objective Optimization

  • 采用NSGA-II算法优化表达式复杂度与拟合误差双目标。
  • 小数据集上性能略有提升,但差异不显著。
  • 避免手动调参,自动平衡模型简洁性与准确性。

Transformation-Interaction-Rational是一种符号回归表示方法,将函数搜索空间限制为两个非线性函数的比值,每个函数是变换变量的线性回归。该方法旨在偏向更简单的表达式,同时保持标准方法的逼近能力。使用遗传编程结合此表示在当代符号回归基准上表现优于其前身(Interaction-Transformation),接近最先进水平。进一步分析发现,在数据量较少时,若对表达式长度施加选择压力,性能可进一步提升。引入适应度函数中的惩罚项有效改善了小数据集结果。但该方法需额外设置两个超参数:惩罚激活条件及惩罚强度。本文将Transformation-Interaction-Rational扩展至支持多目标优化(具体采用NSGA-II算法),并在相同基准上进行测试。结果表明,多目标优化在部分基准上提升了整体性能,其余数据集上表现与单目标方法相当。尤其在小数据集上,仅观察到轻微(且统计上不显著)改进,说明仍需探索更优策略。

原文摘要 · Abstract (English)

The Transformation-Interaction-Rational is a representation for symbolic regression that limits the search space of functions to the ratio of two nonlinear functions each one defined as the linear regression of transformed variables. This representation has the main objective to bias the search towards simpler expressions while keeping the approximation power of standard approaches. The performance of using Genetic Programming with this representation was substantially better than with its predecessor (Interaction-Transformation) and ranked close to the state-of-the-art on a contemporary Symbolic Regression benchmark. On a closer look at these results, we observed that the performance could be further improved with an additional selective pressure for smaller expressions when the dataset contains just a few data points. The introduction of a penalization term applied to the fitness measure improved the results on these smaller datasets. One problem with this approach is that it introduces two additional hyperparameters: i) a criteria to when the penalization should be activated and, ii) the amount of penalization to the fitness function. In this paper, we extend Transformation-Interaction-Rational to support multi-objective optimization, specifically the NSGA-II algorithm, and apply that to the same benchmark. A detailed analysis of the results show that the use of multi-objective optimization benefits the overall performance on a subset of the benchmarks while keeping the results similar to the single-objective approach on the remainder of the datasets. Specifically to the small datasets, we observe a small (and statistically insignificant) improvement of the results suggesting that further strategies must be explored.

符号回归多目标优化过拟合

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。