用共享函数+线性变换,让KAN网络更省参数且性能更强。
GS-KAN: Parameter-Efficient Kolmogorov-Arnold Networks via Sprecher-Type Shared Basis Functions
- 用单一共享函数经可学习变换生成每条边的函数。
- 在连续函数逼近上优于MLP和传统KAN,参数量减少50%以上。
- 适合高维任务中参数受限场景,如大模型轻量化部署。
Kolmogorov-Arnold表示定理为多层感知机(MLP)提供理论替代方案,将可学习的一元函数置于边而非节点上。尽管近年实现的KAN展现出强大逼近能力,但因每条边需独立参数化而存在显著参数效率问题。本文提出GS-KAN(广义Sprecher-KAN),受David Sprecher对叠加定理改进启发,通过在每层使用一个可学习的共享父函数,并施加可学习线性变换生成各边函数,构建独特边函数。我们在合成函数逼近、表格数据回归与图像分类任务上评估了GS-KAN,结果表明:在连续函数逼近任务中,其性能超越MLP与标准KAN基线,同时保持卓越参数效率;在表格回归任务上表现与现有KAN相当,在高维分类任务中优于MLP。关键优势在于,该架构使基于KAN的模型能在严格参数约束下应用于高维场景,而传统实现在此类条件下通常不可行。代码已开源:https://github.com/rambamn48/gs-impl。
原文摘要 · Abstract (English)
The Kolmogorov-Arnold representation theorem offers a theoretical alternative to Multi-Layer Perceptrons (MLPs) by placing learnable univariate functions on edges rather than nodes. While recent implementations such as Kolmogorov-Arnold Networks (KANs) demonstrate high approximation capabilities, they suffer from significant parameter inefficiency due to the requirement of maintaining unique parameterizations for every network edge. In this work, we propose GS-KAN (Generalized Sprecher-KAN), a lightweight architecture inspired by David Sprecher's refinement of the superposition theorem. GS-KAN constructs unique edge functions by applying learnable linear transformations to a single learnable, shared parent function per layer. We evaluate GS-KAN against existing KAN architectures and MLPs across synthetic function approximation, tabular data regression and image classification tasks. Our results demonstrate that GS-KAN outperforms both MLPs and standard KAN baselines on continuous function approximation tasks while maintaining superior parameter efficiency. Additionally, GS-KAN achieves competitive performance with existing KAN architectures on tabular regression and outperforms MLPs on high-dimensional classification tasks. Crucially, the proposed architecture enables the deployment of KAN-based architectures in high-dimensional regimes under strict parameter constraints, a setting where standard implementations are typically infeasible due to parameter explosion. The source code is available at https://github.com/rambamn48/gs-impl.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。