PowerMLP让KAN快40倍，还更准。
PowerMLP: An Efficient Version of KAN
- 用非迭代的简化样条函数替代KAN的复杂计算
- 训练速度比KAN快40倍，精度更高
- 适合需要高速训练的函数拟合与PDE求解任务
Kolmogorov-Arnold网络（KAN）因其在函数拟合和偏微分方程（PDE）求解等任务中的高精度而受到关注。其强大表达能力源于Kolmogorov-Arnold表示定理和可学习样条函数。然而，样条函数的计算涉及多次迭代，导致KAN比MLP慢得多，训练成本显著上升。KAN作者也指出：‘KAN最大的瓶颈在于训练慢，相同参数量下通常比MLP慢10倍。’为此，我们提出一种新型MLP型网络PowerMLP，采用更简单的非迭代样条函数表示，在理论上有更强的表达能力，同时训练时间接近MLP。我们对比了KAN与PowerMLP的浮点运算量（FLOPs），量化了其更快的计算速度。全面实验表明，PowerMLP在多种任务中普遍达到更高精度，且训练速度约为KAN的40倍。
原文摘要 · Abstract (English)
The Kolmogorov-Arnold Network (KAN) is a new network architecture known for its high accuracy in several tasks such as function fitting and PDE solving. The superior expressive capability of KAN arises from the Kolmogorov-Arnold representation theorem and learnable spline functions. However, the computation of spline functions involves multiple iterations, which renders KAN significantly slower than MLP, thereby increasing the cost associated with model training and deployment. The authors of KAN have also noted that ``the biggest bottleneck of KANs lies in its slow training. KANs are usually 10x slower than MLPs, given the same number of parameters.'' To address this issue, we propose a novel MLP-type neural network PowerMLP that employs simpler non-iterative spline function representation, offering approximately the same training time as MLP while theoretically demonstrating stronger expressive power than KAN. Furthermore, we compare the FLOPs of KAN and PowerMLP, quantifying the faster computation speed of PowerMLP. Our comprehensive experiments demonstrate that PowerMLP generally achieves higher accuracy and a training speed about 40 times faster than KAN in various tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。