提出自由节点KAN,提升稳定性并减少参数量。
Free-Knots Kolmogorov-Arnold Network: On the Analysis of Spline Knots and Advancing Stability
- 用样条节点分析KAN,推导出节点数上下界。
- 新方法参数量与MLP相当,训练更稳定。
- 适合需要高效稳定网络的多领域任务。
Kolmogorov-Arnold神经网络(KANs)在机器学习领域受到广泛关注,但其实现常面临训练不稳定和可训练参数过多的问题,且对基于B样条学习到的激活函数行为理解有限。本文从样条节点角度分析KAN,推导出基于B样条的KAN中节点数的下界和上界。为解决现有问题,提出一种新型自由节点KAN,在提升原KAN性能的同时,将可训练参数数量降至与标准多层感知机(MLP)相当的水平。此外,引入新的训练策略,确保可学习样条具有$C^2$连续性,使激活函数更平滑,通过范围扩展改善训练稳定性。所提方法在8个涵盖图像、文本、时间序列、多模态及函数逼近等领域的数据集上进行全面评估,结果表明基于KAN的网络具备可行性,且所提方法有效。
原文摘要 · Abstract (English)
Kolmogorov-Arnold Neural Networks (KANs) have gained significant attention in the machine learning community. However, their implementation often suffers from poor training stability and heavy trainable parameter. Furthermore, there is limited understanding of the behavior of the learned activation functions derived from B-splines. In this work, we analyze the behavior of KANs through the lens of spline knots and derive the lower and upper bound for the number of knots in B-spline-based KANs. To address existing limitations, we propose a novel Free Knots KAN that enhances the performance of the original KAN while reducing the number of trainable parameters to match the trainable parameter scale of standard Multi-Layer Perceptrons (MLPs). Additionally, we introduce new a training strategy to ensure $C^2$ continuity of the learnable spline, resulting in smoother activation compared to the original KAN and improve the training stability by range expansion. The proposed method is comprehensively evaluated on 8 datasets spanning various domains, including image, text, time series, multimodal, and function approximation tasks. The promising results demonstrates the feasibility of KAN-based network and the effectiveness of proposed method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。