提出硬件友好的计算复杂度评估方法,让KAN在部署时更省资源。
Hardware-Oriented Inference Complexity of Kolmogorov-Arnold Networks
- 用乘法、位操作等通用指标评估KAN硬件开销,无需具体芯片设计。
- 覆盖B样条、高斯径向基等四种KAN变体,支持跨平台比较。
- 适合做低延迟、低功耗部署的工程师和算法设计者参考。
Kolmogorov-Arnold网络(KAN)在多种机器学习任务中表现出强大性能,但其独特结构带来显著计算开销。现有研究多基于GPU训练与推理的浮点运算量(FLOPs)评估复杂度,但在光学通信非线性补偿、无线信道状态估计等对延迟和功耗敏感的场景中,训练通常离线完成,推理依赖专用硬件加速器而非GPU。近期硬件实现研究采用查找表(LUT)、寄存器(Flip-Flops)、块存储(Block RAM)等平台相关资源指标,但需完整硬件设计与综合流程,难以用于早期架构决策与跨平台比较。为此,本文推导出适用于各类硬件平台的通用公式,以实数乘法(RM)、位操作(BOP)和加法/位移次数(NABS)量化KAN推理复杂度。分析扩展至多种变体,包括B样条、高斯径向基函数(GRBF)、切比雪夫及傅里叶KAN。所提指标可直接由网络结构计算得出,实现与传统神经网络架构间公平、简便的推理复杂度对比。
原文摘要 · Abstract (English)
Kolmogorov-Arnold Networks (KANs) have recently emerged as a powerful architecture for various machine learning applications. However, their unique structure raises significant concerns regarding their computational overhead. Existing studies primarily evaluate KAN complexity in terms of Floating-Point Operations (FLOPs) required for GPU-based training and inference. However, in many latency-sensitive and power-constrained deployment scenarios, such as neural network-driven non-linearity mitigation in optical communications or channel state estimation in wireless communications, training is performed offline and dedicated hardware accelerators are preferred over GPUs for inference. Recent hardware implementation studies report KAN complexity using platform-specific resource consumption metrics, such as Look-Up Tables, Flip-Flops, and Block RAMs. However, these metrics require a full hardware design and synthesis stage that limits their utility for early-stage architectural decisions and cross-platform comparisons. To address this, we derive generalized, platform-independent formulae for evaluating the hardware inference complexity of KANs in terms of Real Multiplications (RM), Bit Operations (BOP), and Number of Additions and Bit-Shifts (NABS). We extend our analysis across multiple KAN variants, including B-spline, Gaussian Radial Basis Function (GRBF), Chebyshev, and Fourier KANs. The proposed metrics can be computed directly from the network structure and enable a fair and straightforward inference complexity comparison between KAN and other neural network architectures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。