arXiv:2410.14951cs.AI2024-10被引 8

提出单参数KAN,用架构扩展替代复杂基函数,显著提升效率与性能。

Architectural Scaling Surpass Basis Complexity? Efficient KANs with Single-Parameter Design

  • 通过架构扩展替代基函数复杂度,实现高效网络设计。
  • 相比现有KAN,F1提升最高6.51%,测试损失降低93.1%,训练快6倍。
  • 适合追求轻量化、高性能模型的开发者与研究者使用。

Kolmogorov-Arnold Networks(KANs)发展迅速,但缺乏统一理论框架和高效架构设计准则。本文提出三大贡献:首先,构建通用KAN(Uni-KAN)框架,以密集与稀疏表示统一各类KAN结构,并证明其可互换性,开源库支持后续研究;其次,提出高效KAN扩展(EKE)假设,主张将参数投入架构扩展而非基函数复杂度,以获得更好性能;最后,设计单参数KAN(SKAN),实现超轻量级网络。实验首次提供强实证支持:基函数平滑性对稳定训练至关重要。SKAN在多项任务中表现优异,F1最高提升6.51%,测试损失下降93.1%,训练速度提升最多6倍。该工作建立理论框架、指导原则与实用方法,推动下一代高效神经网络发展。代码已公开于https://anonymous.4open.science/r/SKAN-EBBB/。

原文摘要 · Abstract (English)

The landscape of Kolmogorov-Arnold Networks (KANs) is rapidly expanding, yet lacks a unified theoretical framework and a clear principle for efficient architecture design. This paper addresses these gaps with three core contributions. First, we introduce the Universal KAN (Uni-KAN) framework, a novel abstraction that formally unifies all KAN-style networks through dense and sparse representations. We prove their interchangeability and provide an open-source library for this framework, facilitating future research. Second, we propose the Efficient KAN Expansion (EKE) Hypothesis, a design philosophy positing that allocating parameters to architectural scaling rather than basis function complexity yields superior performance. Third, we present Single-Parameter KANs (SKANs), a family of ultra-lightweight networks that embody the EKE Hypothesis. Our comprehensive experiments provide the first strong empirical validation for the theoretical necessity of basis function smoothness for stable training. Furthermore, SKANs demonstrate state-of-the-art performance, improving F1 scores by up to 6.51\% and reducing test loss by 93.1\%, while achieving up to 6x faster training speeds compared to existing KAN variants. These results establish a robust framework, a guiding hypothesis, and a practical methodology for designing the next generation of efficient and powerful neural networks. The code is accessible at https://anonymous.4open.science/r/SKAN-EBBB/.

KAN轻量化架构设计高效训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。