提出两种新初始化方法，显著提升KAN网络性能。
Initialization Schemes for Kolmogorov-Arnold Networks: An Empirical Study
- 基于理论设计两类初始化方案，结合LeCun与Glorot思想。
- 幂律初始化在多任务、多尺寸模型中表现最佳。
- 适合研究KAN架构或需高可解释性的机器学习应用者。
Kolmogorov-Arnold网络（KANs）是一类新型神经架构，用可训练的激活函数替代固定非线性，具有更强灵活性与可解释性。尽管KAN已在科学计算与机器学习任务中取得成功，其初始化策略仍缺乏系统研究。本文针对基于样条的KAN，提出两种受理论启发的初始化方法（分别源自LeCun与Glorot），以及一个可调指数的幂律家族初始化。通过大规模网格搜索，在函数拟合与前向偏微分方程（PDE）基准上评估，并结合神经正切核（NTK）分析训练动态，以及在费曼数据集子集上的测试，结果表明：基于Glorot的初始化在参数丰富的模型中显著优于基线；而幂律初始化整体表现最优，跨任务与不同规模架构均具优势。所有代码与数据已公开于https://github.com/srigas/KAN_Initialization_Schemes。
原文摘要 · Abstract (English)
Kolmogorov-Arnold Networks (KANs) are a recently introduced neural architecture that replace fixed nonlinearities with trainable activation functions, offering enhanced flexibility and interpretability. While KANs have been applied successfully across scientific and machine learning tasks, their initialization strategies remain largely unexplored. In this work, we study initialization schemes for spline-based KANs, proposing two theory-driven approaches inspired by LeCun and Glorot, as well as an empirical power-law family with tunable exponents. Our evaluation combines large-scale grid searches on function fitting and forward PDE benchmarks, an analysis of training dynamics through the lens of the Neural Tangent Kernel, and evaluations on a subset of the Feynman dataset. Our findings indicate that the Glorot-inspired initialization significantly outperforms the baseline in parameter-rich models, while power-law initialization achieves the strongest performance overall, both across tasks and for architectures of varying size. All code and data accompanying this manuscript are publicly available at https://github.com/srigas/KAN_Initialization_Schemes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。