解析遗传编程符号回归的泛化能力,揭示结构与常数优化的理论影响。
On the Generalization Bounds of Symbolic Regression with Genetic Programming
- 从表达式树结构出发,构建带大小、深度和常数约束的泛化界
- 分解泛化误差为结构选择与常数拟合两部分,解释实际设计原理
- 为简洁性压力、深度限制等实践提供理论依据,适合算法设计者
基于遗传编程(GP)的符号回归(SR)旨在从数据中直接发现可解释的数学表达式。尽管其在实践中表现优异,但对为何能泛化到训练数据之外的理解仍不足。本文对以表达式树形式表示的SR模型进行学习理论分析,推导出在树大小、深度及可学习常数受限条件下的泛化界。该结果将泛化误差分解为两个可解释成分:结构选择项(反映表达式树结构的组合复杂度)与常数拟合项(捕捉固定结构内数值常数优化的复杂度)。这一分解为广泛使用的实践(如简洁性压力、深度限制、数值稳定算子、区间算术)提供了理论视角。特别地,分析表明结构限制可降低假设空间增长,而稳定性机制则控制参数扰动对预测的影响。通过将这些设计选择与泛化界中的显式复杂度项关联,本工作为GP符号回归的常见经验行为提供了原理性解释,并推动了对其泛化性质的更严格理解。
原文摘要 · Abstract (English)
Symbolic regression (SR) with genetic programming (GP) aims to discover interpretable mathematical expressions directly from data. Despite its strong empirical success, the theoretical understanding of why GP-based SR generalizes beyond the training data remains limited. In this work, we provide a learning-theoretic analysis of SR models represented as expression trees. We derive a generalization bound for GP-style SR under constraints on tree size, depth, and learnable constants. Our result decomposes the generalization gap into two interpretable components: a structure-selection term, reflecting the combinatorial complexity of choosing an expression-tree structure, and a constant-fitting term, capturing the complexity of optimizing numerical constants within a fixed structure. This decomposition provides a theoretical perspective on several widely used practices in GP, including parsimony pressure, depth limits, numerically stable operators, and interval arithmetic. In particular, our analysis shows how structural restrictions reduce hypothesis-class growth while stability mechanisms control the sensitivity of predictions to parameter perturbations. By linking these practical design choices to explicit complexity terms in the generalization bound, our work offers a principled explanation for commonly observed empirical behaviors in GP-based SR and contributes towards a more rigorous understanding of its generalization properties.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。