用矩阵运算加速高阶B样条计算，让KAN网络训练快40倍。
MatrixKAN: Parallelized Kolmogorov-Arnold Network
- 将B样条计算转为矩阵操作，实现并行化处理。
- 在高阶样条下，计算速度比原KAN快约40倍。
- 适合需要高速训练且使用高阶样条的模型设计者。
Kolmogorov-Arnold网络（KAN）是一类新兴神经网络架构，作为多层感知机（MLP）的有前途替代方案，展现出更强的表达能力和可解释性。然而，由于底层B样条计算具有递归特性，KAN在训练和推理速度上远慢于MLP，尤其在使用高阶B样条时更为明显，因为所需非并行递归次数与样条阶数成正比。本文提出MatrixKAN，通过矩阵表示与运算实现B样条计算的并行化，显著提升高阶样条模型的有效计算效率。实验表明，MatrixKAN的计算时间随样条阶数增长表现更优，相较于原KAN实现约40倍的速度提升，并在更大数据集或更高样条阶数下具备更大提速潜力。
原文摘要 · Abstract (English)
Kolmogorov-Arnold Networks (KAN) are a new class of neural network architecture representing a promising alternative to the Multilayer Perceptron (MLP), demonstrating improved expressiveness and interpretability. However, KANs suffer from slow training and inference speeds relative to MLPs due in part to the recursive nature of the underlying B-spline calculations. This issue is particularly apparent with respect to KANs utilizing high-degree B-splines, as the number of required non-parallelizable recursions is proportional to B-spline degree. We solve this issue by proposing MatrixKAN, a novel optimization that parallelizes B-spline calculations with matrix representation and operations, thus significantly improving effective computation time for models utilizing high-degree B-splines. In this paper, we demonstrate the superior scaling of MatrixKAN's computation time relative to B-spline degree. Further, our experiments demonstrate speedups of approximately 40x relative to KAN, with significant additional speedup potential for larger datasets or higher spline degrees.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。