用截断幂函数改进KAN网络,提升效率与可解释性。
TruKAN: Towards More Efficient Kolmogorov-Arnold Networks Using Truncated Power Functions
- 用截断幂函数替代B样条,简化基函数结构。
- 在视觉任务中实现更高精度、更快训练和更低内存占用。
- 适合追求高效且可解释神经网络的开发者使用。
为解决计算效率与柯尔莫戈洛夫-阿诺德网络(KAN)原理遵循之间的权衡,本文提出TruKAN,一种基于KAN结构和可学习激活函数的新架构。TruKAN将KAN中的B样条基替换为源自k阶样条理论的截断幂函数族,保持表达能力的同时提升准确率与训练速度。每个TruKAN层结合截断幂项与多项式项,并采用共享或独立的节点(knots)。由于基函数与节点配置更简洁,TruKAN具有更强可解释性。我们构建TruKAN模型并集成至高效Net-V2框架,在计算机视觉基准数据集上评估。为公平比较,设计MLP、KAN、SineKAN与TruKAN-based EfficientNet模型,测试其在小型与深层架构下的训练时间与准确率。训练采用混合优化以增强收敛稳定性,并研究层归一化对各模型的影响,评估共享与独立节点在TruKAN中的作用。总体而言,TruKAN在复杂视觉任务中优于其他KAN模型,在准确率、计算效率与内存占用方面表现更优,展现出超越先前研究局限性的优势。
原文摘要 · Abstract (English)
To address the trade-off between computational efficiency and adherence to Kolmogorov-Arnold Network (KAN) principles, we propose TruKAN, a new architecture based on the KAN structure and learnable activation functions. TruKAN replaces the B-spline basis in KAN with a family of truncated power functions derived from k-order spline theory. This change maintains the KAN's expressiveness while enhancing accuracy and training time. Each TruKAN layer combines a truncated power term with a polynomial term and employs either shared or individual knots. TruKAN exhibits greater interpretability than other KAN variants due to its simplified basis functions and knot configurations. By prioritizing interpretable basis functions, TruKAN aims to balance approximation efficacy with transparency. We develop the TruKAN model and integrate it into an advanced EfficientNet-V2-based framework, which is then evaluated on computer vision benchmark datasets. To ensure a fair comparison, we develop various models: MLP-, KAN-, SineKAN and TruKAN-based EfficientNet frameworks and assess their training time and accuracy across small and deep architectures. The training phase uses hybrid optimization to improve convergence stability. Additionally, we investigate layer normalization techniques for all the models and assess the impact of shared versus individual knots in TruKAN. Overall, TruKAN outperforms other KAN models in terms of accuracy, computational efficiency and memory usage on the complex vision task, demonstrating advantages beyond the limited settings explored in prior KAN studies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。