通过分析神经切线核,提升小波型KAN对高频分量的拟合能力。
On the study of frequency control and spectral bias in Wavelet-Based Kolmogorov Arnold networks: A path to physics-informed KANs
- 基于小波基频率与神经切线核的理论分析,调控网络对高频成分的学习
- 实验证明可有效缓解谱偏差,提升对多频物理问题的求解精度
- 适合研究偏微分方程求解、物理信息神经网络的科研人员参考
谱偏差指神经网络在训练初期倾向于优先学习函数的低频成分,这在需要高精度捕捉高频细节的物理信息神经网络(PINNs)中尤为突出。尽管小波型柯尔莫哥洛夫-阿诺德网络(Wav-KANs)在同时建模高低频成分方面表现出色,其理论基础仍不充分,尤其母小波频率与谱偏差之间的关系尚未被深入理解。本文通过分析Wav-KAN的神经切线核(NTK)特征值,揭示其对高频项的收敛机制,从而有效缓解谱偏差。数值实验验证了该方法在解决抛物型、椭圆型和双曲型微分方程中的多频振荡现象时,优于传统PINNs与傅里叶特征方法,展现出更强的适应性。
原文摘要 · Abstract (English)
Spectral bias, the tendency of neural networks to prioritize learning low-frequency components of functions during the initial training stages, poses a significant challenge when approximating solutions with high-frequency details. This issue is particularly pronounced in physics-informed neural networks (PINNs), widely used to solve differential equations that describe physical phenomena. In the literature, contributions such as Wavelet Kolmogorov Arnold Networks (Wav-KANs) have demonstrated promising results in capturing both low- and high-frequency components. Similarly, Fourier features (FF) are often employed to address this challenge. However, the theoretical foundations of Wav-KANs, particularly the relationship between the frequency of the mother wavelet and spectral bias, remain underexplored. A more in-depth understanding of how Wav-KANs manage high-frequency terms could offer valuable insights for addressing oscillatory phenomena encountered in parabolic, elliptic, and hyperbolic differential equations. In this work, we analyze the eigenvalues of the neural tangent kernel (NTK) of Wav-KANs to enhance their ability to converge on high-frequency components, effectively mitigating spectral bias. Our theoretical findings are validated through numerical experiments, where we also discuss the limitations of traditional approaches, such as standard PINNs and Fourier features, in addressing multi-frequency problems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。