arXiv:2510.07570cs.LG2025-10被引 1

用扩散模型一次性生成公式,比传统逐字生成更优。

Symbolic-Diffusion: Deep Learning Based Symbolic Regression with D3PM Discrete Token Diffusion

  • 采用离散扩散模型同时生成公式所有符号
  • 在相同架构下性能接近且部分指标超越自回归模型
  • 适合想提升公式生成质量的研究者

符号回归旨在寻找能拟合数据点的闭式数学表达式。传统方法多基于遗传编程,近年神经网络方法兴起。主流神经网络模型使用基于Transformer的自回归模型,按顺序生成公式符号。但自回归生成受限于前后依赖,无法并行。为此,我们提出Symbolic Diffusion,一种基于D3PM的离散状态空间扩散模型,可一次性生成整个公式的符号。在为SymbolicGPT构建的双变量数据集上,我们对比了该扩散模型与SymbolicGPT的自回归模型,两者采用相同编码器与Transformer架构。结果表明,扩散生成方法在等价架构下表现相当,部分指标更优,为神经网络符号回归开辟新方向。

原文摘要 · Abstract (English)

Symbolic regression refers to the task of finding a closed-form mathematical expression to fit a set of data points. Genetic programming based techniques are the most common algorithms used to tackle this problem, but recently, neural-network based approaches have gained popularity. Most of the leading neural-network based models used for symbolic regression utilize transformer-based autoregressive models to generate an equation conditioned on encoded input points. However, autoregressive generation is limited to generating tokens left-to-right, and future generated tokens are conditioned only on previously generated tokens. Motivated by the desire to generate all tokens simultaneously to produce improved closed-form equations, we propose Symbolic Diffusion, a D3PM based discrete state-space diffusion model which simultaneously generates all tokens of the equation at once using discrete token diffusion. Using the bivariate dataset developed for SymbolicGPT, we compared our diffusion-based generation approach to an autoregressive model based on SymbolicGPT, using equivalent encoder and transformer architectures. We demonstrate that our novel approach of using diffusion-based generation for symbolic regression can offer comparable and, by some metrics, improved performance over autoregressive generation in models using similar underlying architectures, opening new research opportunities in neural-network based symbolic regression.

符号回归扩散模型公式生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。