用物理数据发现的数学激活函数,让生态模型更省参数且不丢精度。
Neuro-Symbolic Activation Discovery: Transferring Mathematical Structures from Physics to Ecology for Parameter-Efficient Neural Networks
- 通过遗传编程从物理数据中挖掘可解释的数学公式作为自定义激活函数。
- 在森林覆盖分类任务上仅用5825个参数达到82.4%准确率,参数量减少5.5倍。
- 揭示了物理到生态的跨领域迁移有效,但物理到文本则无效。
现代神经网络依赖通用激活函数(如ReLU、GELU、SiLU),忽视科学数据中的内在数学结构。我们提出神经符号激活发现框架,利用遗传编程从数据中提取可解释的数学公式,并注入为定制激活函数。关键贡献是发现几何迁移现象:从粒子物理数据中学到的激活函数能成功泛化至生态分类任务,优于标准激活函数(ReLU、GELU、SiLU)在准确率和参数效率上的表现。在Forest Cover数据集上,混合迁移模型以仅5,825个参数实现82.4%准确率,相较传统重型网络(83.4%准确率,31,801参数)实现5.5倍参数压缩,仅损失1%精度。我们引入参数效率评分(E_param = AUC / log₁₀(Params)),证明轻量化混合架构效率比过度参数化基线高18-21%。关键的是,我们确立边界条件:物理到生态迁移成功(均涉及连续欧氏测量),而物理到文本迁移失败(离散词频需不同数学结构)。本工作为高效科学机器学习开辟了领域专用激活函数库的新路径。
原文摘要 · Abstract (English)
Modern neural networks rely on generic activation functions (ReLU, GELU, SiLU) that ignore the mathematical structure inherent in scientific data. We propose Neuro-Symbolic Activation Discovery, a framework that uses Genetic Programming to extract interpretable mathematical formulas from data and inject them as custom activation functions. Our key contribution is the discovery of a Geometric Transfer phenomenon: activation functions learned from particle physics data successfully generalize to ecological classification, outperforming standard activations (ReLU, GELU, SiLU) in both accuracy and parameter efficiency. On the Forest Cover dataset, our Hybrid Transfer model achieves 82.4% accuracy with only 5,825 parameters, compared to 83.4% accuracy requiring 31,801 parameters for a conventional heavy network -- a 5.5x parameter reduction with only 1% accuracy loss. We introduce a Parameter Efficiency Score ($E_{param} = AUC / \log_{10}(Params)$) and demonstrate that lightweight hybrid architectures consistently achieve 18-21% higher efficiency than over-parameterized baselines. Crucially, we establish boundary conditions: while Physics to Ecology transfer succeeds (both involve continuous Euclidean measurements), Physics to Text transfer fails (discrete word frequencies require different mathematical structures). Our work opens pathways toward domain-specific activation libraries for efficient scientific machine learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。