用输入数据复杂度指导符号回归模型选择，避免人为设定参数。
Data-Informed Model Complexity Metric for Optimizing Symbolic Regression Models
- 通过计算模型输出的海森矩阵秩平均值估算复杂度，仅需3个采样点。
- 结合数据内在维度估计，找到表达能力与精度平衡的最佳复杂度区间。
- 适用于需要泛化能力强的符号回归任务，尤其适合无先验经验的研究者。
从训练良好的演化种群中选择能泛化到测试数据的模型十分困难。本文提出一种实用方法，通过后处理阶段的海森矩阵秩来估算模型复杂度。复杂度通过在少量点（N=3）上平均模型输出的海森矩阵秩来近似，实现高效且准确的秩估计。该方法将模型选择与输入数据复杂度对齐，使用内在维度（ID）估计算法进行计算。基于StackGP系统，我们为彭尼机器学习基准数据集构建了符号回归模型，并采用十二种scikit-dimension库中的方法估计ID，使模型表达力与数据集的内在维度相匹配。所提出的数据驱动复杂度度量找到了理想的复杂度窗口，在不引入用户定义参数（如权重选择中的简约压力）常见偏差的前提下，提升了模型的泛化能力。
原文摘要 · Abstract (English)
Choosing models from a well-fitted evolved population that generalizes beyond training data is difficult. We introduce a pragmatic method to estimate model complexity using Hessian rank for post-processing selection. Complexity is approximated by averaging the model output Hessian rank across a few points (N=3), offering efficient and accurate rank estimates. This method aligns model selection with input data complexity, calculated using intrinsic dimensionality (ID) estimators. Using the StackGP system, we develop symbolic regression models for the Penn Machine Learning Benchmark and employ twelve scikit-dimension library methods to estimate ID, aligning model expressiveness with dataset ID. Our data-informed complexity metric finds the ideal complexity window, balancing model expressiveness and accuracy, enhancing generalizability without bias common in methods reliant on user-defined parameters, such as parsimony pressure in weight selection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。