用组合激活函数让神经网络更准拟合数学表达式
CombU: A Combined Unit Activation for Fitting Mathematical Expressions with Neural Networks
- 在不同层不同维度用多种激活函数组合
- 在4个数据集上16项指标中10项超越现有方法
- 适合需要高精度数学建模的科研与工程场景
激活函数是神经网络的核心，通过引入非线性来逼近复杂数据关系。现有研究多聚焦于设计新函数，但我们发现合理组合已有激活函数也能提升性能。本文提出组合单元激活（CombU），在不同层、不同维度采用不同激活函数，理论上可准确拟合多数数学表达式。在4个数学表达式数据集上的实验表明，相比6种SOTA激活函数算法，CombU在16项指标中有10项表现最优，其余6项排名前三。
原文摘要 · Abstract (English)
The activation functions are fundamental to neural networks as they introduce non-linearity into data relationships, thereby enabling deep networks to approximate complex data relations. Existing efforts to enhance neural network performance have predominantly focused on developing new mathematical functions. However, we find that a well-designed combination of existing activation functions within a neural network can also achieve this objective. In this paper, we introduce the Combined Units activation (CombU), which employs different activation functions at various dimensions across different layers. This approach can be theoretically proven to fit most mathematical expressions accurately. The experiments conducted on four mathematical expression datasets, compared against six State-Of-The-Art (SOTA) activation function algorithms, demonstrate that CombU outperforms all SOTA algorithms in 10 out of 16 metrics and ranks in the top three for the remaining six metrics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。