提升KAN网络训练速度，支持并行计算和FPGA部署
Concurrent training methods for Kolmogorov-Arnold networks: Disjoint datasets and FPGA implementation
- 设计预训练与数据分块策略，打破更新依赖链
- 在多个数据子集上并行训练，再合并模型结果
- 适配FPGA的无除法算法，实现硬件直接部署
本文针对基于牛顿-卡茨马尔兹（NK）方法的科尔莫戈罗夫-阿诺德网络（KAN）训练算法，提出三种协同并发策略：(i) 针对NK更新结构设计的预训练流程；(ii) 在不相交数据子集上并行训练后合并模型；(iii) 专为现场可编程门阵列（FPGA）定制的无除法算法，已在设备上实现并测试。这些改进突破了原有算法串行更新的瓶颈，显著提升训练效率。通过计算实验评估了可扩展性，并与主流MLP工具包进行对比。所有实验结果均可复现，完整源码在线公开。
原文摘要 · Abstract (English)
The present paper introduces concurrency-driven enhancements to the training algorithm for the Kolmogorov-Arnold networks (KANs) that is based on the Newton-Kaczmarz (NK) method. Prior research shows that KANs trained using the NK-based approach outperform classical neural networks (multilayer perceptrons - MLPs) both in terms of accuracy and training time. Up to now, the fundamental limitation of the algorithm has been the sequential computation of the updates - each update depends on the results of the previous step, obstructing parallelisation; even though parallelisation of some parts of the algorithm, such as the evaluation of the basis functions, has already been proposed and tested. However, substantial acceleration is achievable. Three complementary concurrency-driven novel strategies are proposed in the present paper: (i) a pre-training procedure tailored to the NK updates' structure, (ii) training on disjoint subsets of data together with models' merging, and (iii) a division-free customisation of the algorithm for field-programmable gate arrays (FPGAs), which is implemented and tested directly on the device. Computational experiments are used to assess the scalability and to compare the authors' KAN implementation with popular MLP packages. All presented experimental results are fully reproducible, with the complete source codes available online.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。