用快速计算函数提升KAN网络效率，兼顾性能与训练速度
Combinations of Fast Activation and Trigonometric Functions in Kolmogorov-Arnold Networks
- 将ReLU、三角函数等快速函数作为KAN的基函数
- 实验表明性能相当，但训练更快、泛化能力更优
- 适合追求高效推理与低延迟部署的场景
长期以来，许多神经网络基于柯尔莫戈罗夫-阿诺德表示定理（KART）构建，旨在解决希尔伯特第13问题。近年来，基于KART的柯尔莫戈罗夫-阿诺德网络（KANs）受到关注，推动了样条和RBF等多项式函数的应用。然而，这些函数在GPU上支持不足且普及度较低。本文提出在KAN中使用快速计算函数，如ReLU、正弦、余弦和反正切函数作为基组件。通过将这些函数组合嵌入网络结构，旨在提升计算效率。实验结果表明，该方法在保持竞争力性能的同时，显著缩短训练时间并增强泛化能力。
原文摘要 · Abstract (English)
For years, many neural networks have been developed based on the Kolmogorov-Arnold Representation Theorem (KART), which was created to address Hilbert's 13th problem. Recently, relying on KART, Kolmogorov-Arnold Networks (KANs) have attracted attention from the research community, stimulating the use of polynomial functions such as B-splines and RBFs. However, these functions are not fully supported by GPU devices and are still considered less popular. In this paper, we propose the use of fast computational functions, such as ReLU and trigonometric functions (e.g., ReLU, sin, cos, arctan), as basis components in Kolmogorov-Arnold Networks (KANs). By integrating these function combinations into the network structure, we aim to enhance computational efficiency. Experimental results show that these combinations maintain competitive performance while offering potential improvements in training time and generalization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。