用Transformer和遗传算法生成可解释的数学表达式,精准还原数据背后的物理规律。
Decomposable Neural Symbolic Regression
- 通过多集合Transformer提取变量影响的单变量骨架,实现分步可解释建模。
- 在噪声控制与变噪声问题上,外推误差低于或接近主流方法,且表达式结构更贴近真实公式。
- 适合需要模型可解释性的科学计算、物理建模等场景,尤其适用于复杂系统逆向工程。
符号回归(SR)通过发现能捕捉观测数据内在关系的数学表达式来建模复杂系统。然而,多数现有方法侧重于降低预测误差,常导致表达式过于复杂或不准确。为此,本文提出一种可分解的符号回归方法,利用变压器模型、遗传算法(GAs)和遗传编程(GP)生成可解释的多变量表达式。该方法将训练好的“黑箱”回归模型解释为数学表达式,采用多集合变压器生成多个描述各变量影响的单变量符号骨架。随后通过基于遗传算法的方法筛选高质量候选骨架,并通过基于遗传编程的级联过程逐步合并,同时保持原始骨架结构。最终的多变量骨架经遗传算法优化系数。我们在含控噪和变噪的问题上进行评估,结果表明其插值和外推误差低于或接近两种基于遗传编程的方法、三种神经符号回归方法及一种混合方法。与这些方法不同,本方法始终能学习到与原数学结构一致的表达式。在Feynman数据集上,本方法也实现了高符号解恢复率,并在预测性能上达到基准方法水平。
原文摘要 · Abstract (English)
Symbolic regression (SR) models complex systems by discovering mathematical expressions that capture underlying relationships in observed data. However, most SR methods prioritize minimizing prediction error over identifying the governing equations, often producing overly complex or inaccurate expressions. To address this, we present a decomposable SR method that generates interpretable multivariate expressions leveraging transformer models, genetic algorithms (GAs), and genetic programming (GP). In particular, our explainable SR method distills a trained "opaque'' regression model into mathematical expressions that serve as explanations of its computed function. Our method employs a Multi-Set Transformer to generate multiple univariate symbolic skeletons that characterize how each variable influences the opaque model's response. We then evaluate the generated skeletons' performance using a GA-based approach to select a subset of high-quality candidates before incrementally merging them via a GP-based cascade procedure that preserves their original skeleton structure. The final multivariate skeletons undergo coefficient optimization via a GA. We evaluated our method on problems with controlled and varying degrees of noise, demonstrating lower or comparable interpolation and extrapolation errors compared to two GP-based methods, three neural SR methods, and a hybrid approach. Unlike these methods, our approach consistently learned expressions that matched the original mathematical structure. Similarly, our method achieved both a high symbolic solution recovery rate and competitive predictive performance relative to benchmark methods on the Feynman dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。