通过迭代优化降低符号回归复杂度,提升表达式简洁性。
Symbolic Regression via Latent Iterative Refinement

- 在函数空间中构建可迭代优化的潜在表示,实现逐轮修正。
- 在三个噪声水平下,表达式复杂度低至8-11,比最强基线低2-10倍。
- 适合追求简洁高精度数学表达式的科研与工程应用。
符号回归(SR)旨在寻找拟合观测数据的闭式数学表达式。神经符号回归方法通过训练编码器将观测直接映射为表达式,实现单次推断,但这种近似推理存在残余泛化差距。本文提出潜在方程嵌入(LEE)框架,通过函数意义驱动的潜在空间中的迭代推理填补该差距。LEE学习一个共享潜在空间Z,包含三个组件:联合嵌入符号标记与数值观测的编码器f_theta;从潜在向量重建公式的表达式解码器g_expr;以及从潜在向量预测函数值的评估解码器g_eval,显式地将潜在空间与函数行为对齐。推理时,LEE通过重新编码解码出的表达式与观测数据进行迭代修正,逐步优化潜在估计。同时利用编码器作为学习到的优化器,每轮重编码隐式计算候选表达式与数据的不匹配。由于g_eval在潜在变量上可微,进一步融合连续梯度下降与离散重编码,形成混合迭代与梯度优化流程。在SRBench数据集上,面对三种噪声水平,相较19种基线(涵盖遗传编程、符号神经混合模型及预训练Transformer),LEE生成的表达式复杂度为8–11,远低于基线的20–90,简洁性提升2–10倍。结果推动了准确率-复杂度帕累托前沿向低复杂度方向发展,并展现出随噪声增加的平稳退化特性。
原文摘要 · Abstract (English)
Symbolic regression (SR) seeks closed-form mathematical expressions that fit observed data. Neural SR methods amortize the search by training an encoder to map observations directly to expressions in a single pass, but this amortized inference leaves a residual amortization gap between its one-shot prediction and the true posterior. We propose Latent Equation Embedding (LEE), a framework that closes this gap through iterative amortized inference in a functionally grounded latent space. LEE learns a shared latent space Z equipped with three components: an encoder f_theta that jointly embeds symbolic tokens and numerical observations into a single latent vector z; an expression decoder g_expr that reconstructs formulas from z; and an evaluation decoder g_eval that predicts function values from z, explicitly grounding the latent space in functional behavior. At inference, LEE performs iterative refinement by re-encoding decoded expressions jointly with observations, progressively improving the latent estimate. LEE uses the encoder itself as a learned inference optimizer: each re-encoding step implicitly computes the mismatch between the candidate and the data. Because g_eval is differentiable in z, we additionally interleave continuous gradient descent with discrete re-encoding, yielding a hybrid iterative and gradient refinement procedure. On SRBench across three noise levels, against 19 baselines spanning genetic programming, symbolic-neural hybrids, and pre-trained Transformers, LEE produces expressions 2--10x simpler than the strongest accuracy-oriented baselines, including Operon, GP-GOMEA, TPSR, RAG-SR, and GenSR, with complexity 8--11 versus 20--90. These results advance the low-complexity region of the accuracy-complexity Pareto frontier and show graceful degradation as noise increases.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。