arXiv:2505.06091cs.LGcs.AI2025-05

用Transformer引导符号网络,提升回归表达式发现效率与精度。

UniSymNet: A Unified Symbolic Network Guided by Transformer

  • 将二元非线性运算统一为嵌套一元运算,简化结构复杂度。
  • 在低维标准数据集和高维SRBench上均实现高拟合精度与低复杂度表达式。
  • 通过预训练Transformer指导结构选择,适合需要简洁数学表达的科研场景。

符号回归(SR)是一种从数据中自动发现数学表达式的强大技术。主流SR算法在庞大的函数空间中搜索最优符号树,但树结构日益复杂限制了性能。受神经网络启发,符号网络成为新范式,但多数仍存在缺陷:二元非线性算子{×, ÷}难以自然扩展至多元,且固定架构训练易导致复杂度高与过拟合。本文提出统一符号网络UniSymNet,将二元非线性算子统一为嵌套一元算子,并定义其降低复杂度的条件。同时,采用新型标签编码方法预训练Transformer模型以指导结构选择,并结合任务特异性优化策略学习符号网络参数。UniSymNet在低维标准基准和高维SRBench上均表现出高拟合精度、优秀符号解率及相对低表达复杂度,性能具有竞争力。

原文摘要 · Abstract (English)

Symbolic Regression (SR) is a powerful technique for automatically discovering mathematical expressions from input data. Mainstream SR algorithms search for the optimal symbolic tree in a vast function space, but the increasing complexity of the tree structure limits their performance. Inspired by neural networks, symbolic networks have emerged as a promising new paradigm. However, most existing symbolic networks still face certain challenges: binary nonlinear operators $\{\times, ÷\}$ cannot be naturally extended to multivariate operators, and training with fixed architecture often leads to higher complexity and overfitting. In this work, we propose a Unified Symbolic Network that unifies nonlinear binary operators into nested unary operators and define the conditions under which UniSymNet can reduce complexity. Moreover, we pre-train a Transformer model with a novel label encoding method to guide structural selection, and adopt objective-specific optimization strategies to learn the parameters of the symbolic network. UniSymNet shows high fitting accuracy, excellent symbolic solution rate, and relatively low expression complexity, achieving competitive performance on low-dimensional Standard Benchmarks and high-dimensional SRBench.

符号回归Transformer表达式发现自动化建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。