arXiv:2603.04827cs.LGcs.AI2026-03被引 1

通过分层训练提升KAN网络的训练效率,显著加快收敛速度。

Multilevel Training for Kolmogorov Arnold Networks

  • 利用样条基函数与幂ReLU的等价性,构建可分层优化的网络结构
  • 多级训练在物理信息神经网络中实现精度数量级提升
  • 适合追求高效训练的结构化神经网络研究者

通用神经网络的训练加速受限于其函数复合结构缺乏保障。与多层感知机(MLP)不同,柯尔莫戈洛夫-阿诺德网络(KAN)通过指定基函数展开学习激活值,具有更强结构。本文利用该结构,提出实用算法与理论洞察,实现基于多级训练的训练加速。首先建立具有样条基函数的KAN与带幂ReLU激活的多通道MLP之间的线性基变换等价关系;进而分析该基变换对梯度优化几何的影响,尤其针对样条节点(knots)。由此启发的多级训练策略,通过样条节点的均匀细化构造一系列自然定义的KAN模型,并使用解析几何插值算子连接各层级。该插值方案确保了“恰当嵌套”的架构层次,使向精细模型的插值能保留粗粒度模型的优化成果,而样条基的紧支撑特性则保证后续层级的互补优化。数值实验表明,相比传统方法,该多级训练在同等规模的KAN或MLP上实现数量级的精度提升,尤其在物理信息神经网络中表现突出。本工作展示,合理设计网络结构可带来可利用的数学结构,从而催生显著提升训练性能的多级算法。

原文摘要 · Abstract (English)

Algorithmic speedup of training common neural architectures is made difficult by the lack of structure guaranteed by the function compositions inherent to such networks. In contrast to multilayer perceptrons (MLPs), Kolmogorov-Arnold networks (KANs) provide more structure by expanding learned activations in a specified basis. This paper exploits this structure to develop practical algorithms and theoretical insights, yielding training speedup via multilevel training for KANs. To do so, we first establish an equivalence between KANs with spline basis functions and multichannel MLPs with power ReLU activations through a linear change of basis. We then analyze how this change of basis affects the geometry of gradient-based optimization with respect to spline knots. The KANs change-of-basis motivates a multilevel training approach, where we train a sequence of KANs naturally defined through a uniform refinement of spline knots with analytic geometric interpolation operators between models. The interpolation scheme enables a ``properly nested hierarchy'' of architectures, ensuring that interpolation to a fine model preserves the progress made on coarse models, while the compact support of spline basis functions ensures complementary optimization on subsequent levels. Numerical experiments demonstrate that our multilevel training approach can achieve orders of magnitude improvement in accuracy over conventional methods to train comparable KANs or MLPs, particularly for physics informed neural networks. Finally, this work demonstrates how principled design of neural networks can lead to exploitable structure, and in this case, multilevel algorithms that can dramatically improve training performance.

KAN多级训练神经网络优化样条基

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。