用元学习减少KAN训练参数，提升内存效率。
Improving Memory Efficiency for Training KANs via Meta Learning
- 用小型元学习器生成KAN权重，降低参数量。
- 在符号回归等任务上性能接近甚至优于原版KAN。
- 适合需要高效训练和可解释性的深度学习应用。
受柯尔莫哥洛夫-阿诺德表示定理启发，KAN通过将传统神经网络权重替换为可学习的单变量函数，提供了一种高效且可解释的函数逼近新框架。然而，KAN的可训练参数数量显著增加，导致内存效率低、训练成本高。为此，我们提出通过更小的元学习器——MetaKANs生成KAN权重。通过端到端可微方式联合训练KAN与MetaKANs，MetaKANs在保持良好可解释性的同时，显著减少可训练参数，实现相当或更优的性能。在符号回归、偏微分方程求解及图像分类等多个基准任务上的实验表明，该方法有效提升了参数效率与内存使用表现。所提方法为KAN训练提供了更具扩展性与可扩展性的替代方案，缩小了其与MLP之间的训练成本差距。代码已开源：https://github.com/Murphyzc/MetaKAN。
原文摘要 · Abstract (English)
Inspired by the Kolmogorov-Arnold representation theorem, KANs offer a novel framework for function approximation by replacing traditional neural network weights with learnable univariate functions. This design demonstrates significant potential as an efficient and interpretable alternative to traditional MLPs. However, KANs are characterized by a substantially larger number of trainable parameters, leading to challenges in memory efficiency and higher training costs compared to MLPs. To address this limitation, we propose to generate weights for KANs via a smaller meta-learner, called MetaKANs. By training KANs and MetaKANs in an end-to-end differentiable manner, MetaKANs achieve comparable or even superior performance while significantly reducing the number of trainable parameters and maintaining promising interpretability. Extensive experiments on diverse benchmark tasks, including symbolic regression, partial differential equation solving, and image classification, demonstrate the effectiveness of MetaKANs in improving parameter efficiency and memory usage. The proposed method provides an alternative technique for training KANs, that allows for greater scalability and extensibility, and narrows the training cost gap with MLPs stated in the original paper of KANs. Our code is available at https://github.com/Murphyzc/MetaKAN.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。