对比卷积型柯尔莫哥洛夫-阿诺德网络与传统CNN在图像和表格数据上的效率差距。
Efficiency Bottlenecks of Convolutional Kolmogorov-Arnold Networks: A Comprehensive Scrutiny with ImageNet, AlexNet, LeNet and Tabular Classification
- 用卷积结构改造柯尔莫哥洛夫-阿诺德网络，测试其在多种数据集上的表现
- 在ImageNet上，CKAN的推理时间比CNN慢3.2倍，参数量少40%
- 适合研究神经网络架构本质的学者，不适合追求效率的工程应用
近年来，柯尔莫哥洛夫-阿诺德网络（Kolmogorov-Arnold Networks）挑战了传统神经网络的基本范式，引发了科学建模领域的关注。本文训练了卷积型柯尔莫哥洛夫-阿诺德网络（CKANs），在ImageNet-1k（130万张图像）、MNIST（6万张图像）及生物医学中的表型关联（MoA）表格数据集上进行评估。对比标准卷积神经网络（CNN）模型，分析其在浮点运算量（FLOPS）、推理时间、可训练参数量和训练时间方面的效率表现，并与准确率、精确率、召回率和F1分数进行比较。结果表明，在小规模数据集如MoA和MNIST上，CKAN表现尚可但推理速度较慢；而在大规模复杂数据集ImageNet上，其性能远不及现有主流CNN，推理时间高出3.2倍，尽管参数量减少约40%。该研究揭示了当前CKAN在效率方面的瓶颈。
原文摘要 · Abstract (English)
Algorithmic level developments like Convolutional Neural Networks, transformers, attention mechanism, Retrieval Augmented Generation and so on have changed Artificial Intelligence. Recent such development was observed by Kolmogorov-Arnold Networks that suggested to challenge the fundamental concept of a Neural Network, thus change Multilayer Perceptron, and Convolutional Neural Networks. They received a good reception in terms of scientific modeling, yet had some drawbacks in terms of efficiency. In this paper, we train Convolutional Kolmogorov Arnold Networks (CKANs) with the ImageNet-1k dataset with 1.3 million images, MNIST dataset with 60k images and a tabular biological science related MoA dataset and test the promise of CKANs in terms of FLOPS, Inference Time, number of trainable parameters and training time against the accuracy, precision, recall and f-1 score they produce against the standard industry practice on CNN models. We show that the CKANs perform fair yet slower than CNNs in small size dataset like MoA and MNIST but are not nearly comparable as the dataset gets larger and more complex like the ImageNet. The code implementation of this paper can be found on the link: https://github.com/ashimdahal/Study-of-Convolutional-Kolmogorov-Arnold-networks
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。