用描述长度最小化替代误差最小化,提升公式发现的准确率。
Symbolic regression via MDLformer-guided search: from minimizing prediction error to minimizing description length
- 以最小描述长度为搜索目标,使优化过程单调收敛。
- 在133个基准问题中成功恢复约50个公式,优于现有方法43.92%。
- 适用于需要可解释公式的科学建模与黑箱系统逆向工程。
符号回归旨在从数据中发现最优拟合公式,传统方法依赖启发式搜索,通过迭代更新候选公式降低预测误差。然而,函数形状相近的公式可能具有截然不同的符号形式,导致预测误差无法随逼近目标公式而单调下降,从而造成现有方法恢复率低。为此,本文提出基于最小描述长度(MDL)的新搜索目标,该目标能反映与真实公式的距离,并在搜索过程中单调递减。为此设计了神经网络模型MDLformer,通过大规模训练实现对任意数据集的稳健、可扩展的MDL估计。结合MDLformer输出,构建符号回归方法SR4MDL,可有效恢复正确数学形式。大量实验表明,该方法在包含133个问题的两个基准数据集上成功恢复约50个公式,性能优于当前最先进方法43.92%。在122个未见黑箱问题上的实验进一步验证其泛化能力。代码已开源:https://github.com/tsinghua-fib-lab/SR4MDL。
原文摘要 · Abstract (English)
Symbolic regression, a task discovering the formula best fitting the given data, is typically based on the heuristical search. These methods usually update candidate formulas to obtain new ones with lower prediction errors iteratively. However, since formulas with similar function shapes may have completely different symbolic forms, the prediction error does not decrease monotonously as the search approaches the target formula, causing the low recovery rate of existing methods. To solve this problem, we propose a novel search objective based on the minimum description length, which reflects the distance from the target and decreases monotonically as the search approaches the correct form of the target formula. To estimate the minimum description length of any input data, we design a neural network, MDLformer, which enables robust and scalable estimation through large-scale training. With the MDLformer's output as the search objective, we implement a symbolic regression method, SR4MDL, that can effectively recover the correct mathematical form of the formula. Extensive experiments illustrate its excellent performance in recovering formulas from data. Our method successfully recovers around 50 formulas across two benchmark datasets comprising 133 problems, outperforming state-of-the-art methods by 43.92%. Experiments on 122 unseen black-box problems further demonstrate its generalization performance. We release our code at https://github.com/tsinghua-fib-lab/SR4MDL .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。