arXiv:2507.04690cs.LGcs.AI2025-07被引 6

MJKAN融合KAN与MLP优势,提升模型效率与表达力。

Bridging KAN and MLP: MJKAN, a Hybrid Architecture with Both Efficiency and Expressiveness

  • 采用特征调制机制与RBF激活函数,构建混合架构。
  • 函数回归任务中显著优于MLPs,性能随基函数增加而提升。
  • 小基函数数更利于分类任务泛化,适合需高效表达的场景。

Kolmogorov-Arnold网络(KANs)通过可学习的单变量函数替代固定激活函数,但存在计算成本高、通用分类任务表现不足的问题。本文提出调制联合KAN(MJKAN),一种新型神经网络层,结合特征式线性调制(FiLM)机制与径向基函数(RBF)激活,融合KAN的非线性表达力与MLP的计算效率。在函数回归、图像分类(MNIST、CIFAR-10/100)及自然语言处理(AG News、SMS Spam)等多类基准上验证性能。结果表明,MJKAN在函数回归中显著优于MLPs,且性能随基函数数量增加而提升;而在图像与文本分类中表现与MLPs相当,但对基函数数量敏感——较小基函数数更利于泛化,提示需根据数据复杂度精细调控模型容量以避免过拟合。总体而言,MJKAN兼具KAN理论优势与实际可行性,提供灵活高效的建模方案。

原文摘要 · Abstract (English)

Kolmogorov-Arnold Networks (KANs) have garnered attention for replacing fixed activation functions with learnable univariate functions, but they exhibit practical limitations, including high computational costs and performance deficits in general classification tasks. In this paper, we propose the Modulation Joint KAN (MJKAN), a novel neural network layer designed to overcome these challenges. MJKAN integrates a FiLM (Feature-wise Linear Modulation)-like mechanism with Radial Basis Function (RBF) activations, creating a hybrid architecture that combines the non-linear expressive power of KANs with the efficiency of Multilayer Perceptrons (MLPs). We empirically validated MJKAN's performance across a diverse set of benchmarks, including function regression, image classification (MNIST, CIFAR-10/100), and natural language processing (AG News, SMS Spam). The results demonstrate that MJKAN achieves superior approximation capabilities in function regression tasks, significantly outperforming MLPs, with performance improving as the number of basis functions increases. Conversely, in image and text classification, its performance was competitive with MLPs but revealed a critical dependency on the number of basis functions. We found that a smaller basis size was crucial for better generalization, highlighting that the model's capacity must be carefully tuned to the complexity of the data to prevent overfitting. In conclusion, MJKAN offers a flexible architecture that inherits the theoretical advantages of KANs while improving computational efficiency and practical viability.

神经网络KAN混合架构高效表达

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。