不同架构导致符号回归结果差异,需通过验证集选择最佳架构。
Architecture-Induced Recoverability Bias in Differentiable Symbolic Regression
- 固定语法和训练流程,仅改变变量路由架构比较其影响
- 同一目标下恢复率从0/64到64/64,架构效果差异巨大
- 建议用验证集选出最优架构,而非预先固定
符号回归旨在从数值数据中恢复闭式表达式,但在可微符号回归中,恢复结果不仅依赖于语法,还受固定架构下变量传递路径的影响。该现象在需要可解释非线性结构的信号处理场景中尤为关键。现有研究常将架构与算子族、语法或搜索方法混杂,难以直接分离架构效应。本研究在固定算子族、语法和训练协议的前提下,对比三种深度为3的架构,覆盖24种算子-形状-叶节点组合。结果显示,相同目标下的恢复率在0/64至64/64之间变化;某一架构在某目标表现最佳,却在另一目标最差;具有两个同深子树的树结构在所有测试配置中均失败(0/3,776)。作为概念验证,采用小规模架构集进行训练,选取验证集上均方误差最低的表达式。在联合运行子集上,恢复率从单一架构的34.4%提升至50.1%。在Shockley二极管目标上,验证选择器成功恢复了基线架构遗漏的案例(基线恢复0/32种子)。由于联合子集仅含三个配置,结果表明基于验证的选择策略有潜力,但尚未构成完整基准。研究支持将架构视为可测量的设计变量,应报告、压力测试,并通过保留验证集选择,而非先验固定。
原文摘要 · Abstract (English)
Symbolic regression aims to recover closed-form expressions from numerical data, but in differentiable symbolic regression the recovered expression depends not only on the grammar but also on the fixed architecture through which variables are routed during training. This is relevant to signal-processing settings in which closed-form models and interpretable nonlinear structure are useful. This architecture-specific effect has rarely been isolated directly, because existing comparisons often vary architecture together with operator family, grammar, or search procedure. Three depth-3 architectures are compared across twenty-four operator--shape--leaf combinations, holding operator family, grammar, and training protocol fixed as far as possible while varying the variable-routing architecture. Recovery changes from $0/64$ to $64/64$ trials on the same target under an architecture-plus-native-training-protocol comparison. The best architecture on one target is the worst on another, and trees with two equal-depth subtrees fail in every configuration tested ($0/3{,}776$). As a proof-of-concept mitigation, a small architecture set is trained and the hardened expression with the lowest held-out RMSE is selected. On the jointly-run subset, this improves recovery from $34.4\%$ for the only architecture present in all three configurations to $50.1\%$. On a Shockley diode target, the validation selector recovers cases missed by that baseline architecture, which by itself recovers $0/32$ seeds. Since the jointly-run subset contains only three configurations, the selector result is evidence that validation-based architecture selection is promising, not a complete benchmark. These results support treating architecture as a measurable design variable that should be reported, stress-tested, and selected using held-out validation rather than fixed a priori.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。