PRKAN通过精简参数量，让KAN网络媲美MLP性能。
PRKAN: Parameter-Reduced Kolmogorov-Arnold Networks
- 用多方法压缩KAN层参数，接近MLP规模。
- 在MNIST/Fashion-MNIST上超越多数KAN，部分达MLP水平。
- 适合追求高效高精度神经网络的科研与工程应用。
Kolmogorov-Arnold Networks (KANs) 为神经网络架构带来创新，是CNN、RNN和Transformer等模型中对MLPs的有力替代。但现有KAN通常比MLP需要更多参数。本文提出参数缩减型KAN（PRKAN），通过多种方法降低KAN层参数量，使其与MLP相当。在MNIST和Fashion-MNIST数据集上的实验表明，PRKAN优于多个现有KAN模型，其带注意力机制的变体性能接近甚至媲美MLP，尽管训练时间略长。研究还强调了高斯径向基函数（GRBFs）和层归一化在KAN设计中的优势。代码仓库见：https://github.com/hoangthangta/All-KAN。
原文摘要 · Abstract (English)
Kolmogorov-Arnold Networks (KANs) represent an innovation in neural network architectures, offering a compelling alternative to Multi-Layer Perceptrons (MLPs) in models such as Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and Transformers. By advancing network design, KANs drive groundbreaking research and enable transformative applications across various scientific domains involving neural networks. However, existing KANs often require significantly more parameters in their network layers than MLPs. To address this limitation, this paper introduces PRKANs (Parameter-Reduced Kolmogorov-Arnold Networks), which employ several methods to reduce the parameter count in KAN layers, making them comparable to MLP layers. Experimental results on the MNIST and Fashion-MNIST datasets demonstrate that PRKANs outperform several existing KANs, and their variant with attention mechanisms rivals the performance of MLPs, albeit with slightly longer training times. Furthermore, the study highlights the advantages of Gaussian Radial Basis Functions (GRBFs) and layer normalization in KAN designs. The repository for this work is available at: https://github.com/hoangthangta/All-KAN.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。