arXiv:2502.18959cs.LGstat.ML2025-02被引 6

用正弦基函数构建多组件多层网络,高效捕捉高频信号。

Fourier Multi-Component and Multi-Layer Neural Networks: Unlocking High-Frequency Potential

  • 每层用可训练的正弦基函数线性组合表示组件,逐层生成复杂高频特征。
  • 低秩结构下仍保持指数级函数逼近能力,且优化更稳定。
  • 适合需要高精度建模振荡函数的场景,如物理模拟与信号处理。

神经网络架构与激活函数的选择对其性能至关重要,二者需良好匹配以实现有效表征与学习。本文提出傅里叶多组件多层神经网络(FMMNN),将正弦型激活函数与多组件、多层结构结合。在FMMNN中,每个组件被表示为固定随机正弦基函数的可训练线性组合,多层叠加生成更复杂自适应的高频特征。我们证明,即使在低秩结构下,FMMNN仍具备指数级函数逼近能力。分析表明,其优化景观显著优于标准全连接网络,尤其对高频目标。此外,我们提出一种缩放随机初始化方法,可加速训练并提升性能。大量数值实验验证了理论结果,显示FMMNN在振荡函数逼近基准上表现优异,准确率高且收敛稳定。

原文摘要 · Abstract (English)

The architecture of a neural network and the choice of its activation function are both fundamental to its performance. Equally important is ensuring that these two elements are well matched, as their alignment is key to effective representation and learning. In this paper, we introduce the Fourier Multi-Component and Multi-Layer Neural Network (FMMNN), a model that combines sine-type activations with the multi-component and multi-layer structure of MMNNs. In an FMMNN, each component is represented as a trainable linear combination of fixed random sine-type basis functions, while multi-layer composition generates more complex and adaptive high-frequency features. We establish that FMMNNs retain exponential expressive power for function approximation even under a low-rank architectural structure. We also analyze the optimization landscape of FMMNNs and find it to be substantially more favorable than that of standard fully connected neural networks, especially for high-frequency targets. In addition, we propose a scaled random initialization method for the first-layer weights in FMMNNs, which accelerates training and improves final performance when sufficient samples are available. Extensive numerical experiments support our theoretical insights, showing that FMMNNs achieve strong accuracy and favorable convergence behavior on oscillatory function-approximation benchmarks.

神经网络高频建模正弦激活

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。