用数学定理重构卷积层,让视觉模型更高效可解释。
KAConvNet: Kolmogorov-Arnold Convolutional Networks for Vision Recognition

- 基于柯尔莫哥洛夫-阿诺德定理设计新型卷积层,理论基础扎实。
- 在多个数据集上性能超越现有混合KAN-CNN方法,接近主流ViT/CNN水平。
- 代码开源,适合关注模型可解释性与架构创新的研究者。
卷积神经网络(CNN)一直是通用计算机视觉任务的主流有效方法。近期,基于柯尔莫哥洛夫-阿诺德表示定理的柯尔莫哥洛夫-阿诺德网络(KANs)展现出替代深度学习中多层感知机(MLPs)的潜力。相较于MLPs,KANs通过边上的可学习非线性激活和节点上的简单求和,具有更少参数和更强可解释性。然而,将柯尔莫哥洛夫-阿诺德表示定理与卷积方法结合的研究仍有限,现有尝试仅将可学习激活函数替换为权重,削弱了KANs的理论根基并限制其有效性。此外,KANs中使用的B样条曲线存在计算效率低和易过拟合的问题。本文提出一种新型柯尔莫哥洛夫-阿诺德卷积层,深度整合该定理与卷积机制。该层因基于成熟数学定理,具备更强的方法可解释性和理论一致性。在此基础上,我们构建了高效的KAConvNet网络架构,其性能优于现有混合KAN-CNN方法,并达到与主流视觉变换器(ViTs)和CNNs相当的水平。我们认为本工作为人工智能领域提供了宝贵洞见,将激励2020年代更多创新性CNN的发展。代码已公开于https://github.com/UnicomAI/KAConvNet。
原文摘要 · Abstract (English)
The Convolutional Neural Networks (CNNs) have been the dominant and effective approach for general computer vision tasks. Recently, Kolmogorov-Arnold neural networks (KANs), based on the Kolmogorov-Arnold representation theorem, have shown potential to replace Multi-Layer Perceptrons (MLPs) in deep learning. KANs, which use learnable nonlinear activations on edges and simple summation on nodes, offer fewer parameters and greater explainability compared to MLPs. However, there has been limited exploration of integrating the Kolmogorov-Arnold representation theorem with convolutional methods for computer vision tasks. Existing attempts have merely replaced learnable activation functions with weights, undermining KANs' theoretical foundation and limiting their potential effectiveness. Additionally, the B-spline curves used in KANs suffer from computational inefficiency and a tendency to overfit. In this paper, we propose a novel Kolmogorov-Arnold Convolutional Layer that deeply integrates the Kolmogorov-Arnold representation theorem with convolution. This layer provides stronger method interpretability because it is based on established mathematical theorems and its design has theoretical alignment. Building on the Kolmogorov-Arnold Convolutional Layer, we design an efficient network architecture called KAConvNet, which outperforms existing methods combining KAN and convolution, and achieves competitive performance compared to mainstream ViTs and CNNs. We believe that our work offers valuable insight into the field of artificial intelligence and will inspire the development of more innovative CNNs in the 2020s. The code is publicly available at https://github.com/UnicomAI/KAConvNet.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。