测试时计算可缓解神经符号回归中的复制偏差问题。
Can Test-time Computation Mitigate Reproduction Bias in Neural Symbolic Regression?
- 提出在测试阶段引入额外信息，打破生成表达式对训练数据的依赖。
- 发现传统逐标记生成无法保证数值一致性，导致性能下降。
- 适合关注符号回归鲁棒性与泛化能力的研究者参考。
数学表达式在科学发现中至关重要。符号回归旨在从给定数值数据中自动发现此类表达式。近年来，基于在合成数据集上预训练的Transformer的神经符号回归（NSR）方法因其快速推理而受到关注，但在输入变量较多时表现不佳。本研究从理论和实证角度分析发现：(1) 传统的逐标记生成方式不适用于NSR，因Transformer无法在生成过程中组合性地验证数值一致性；(2) NSR的搜索空间受复制偏差严重限制，多数生成表达式仅来自训练数据的复制。进一步研究发现，测试阶段引入额外信息能有效缓解该偏差。研究成果深化了对NSR局限性的理解，并为设计更鲁棒、泛化的模型提供指导。代码已开源：https://github.com/Shun-0922/Mem-Bias-NSR。
原文摘要 · Abstract (English)
Mathematical expressions play a central role in scientific discovery. Symbolic regression aims to automatically discover such expressions from given numerical data. Recently, Neural symbolic regression (NSR) methods that involve Transformers pre-trained on synthetic datasets have gained attention for their fast inference, but they often perform poorly, especially with many input variables. In this study, we analyze NSR from both theoretical and empirical perspectives and show that (1) ordinary token-by-token generation is ill-suited for NSR, as Transformers cannot compositionally generate tokens while validating numerical consistency, and (2) the search space of NSR methods is greatly restricted due to reproduction bias, where the majority of generated expressions are merely copied from the training data. We further examine whether tailored test-time strategies can reduce reproduction bias and show that providing additional information at test time effectively mitigates it. These findings contribute to a deeper understanding of the limitation of NSR approaches and provide guidance for designing more robust and generalizable methods. Code is available at https://github.com/Shun-0922/Mem-Bias-NSR .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。