arXiv:2508.00247stat.MLcs.LG2025-08被引 3

用可学习频率的正弦函数构建新型KAN,理论有效且性能媲美MLP。

Sinusoidal Approximation Theorem for Kolmogorov-Arnold Networks

  • 用可学习频率的正弦函数替代KAN中的样条基函数
  • 在多个多变量函数上优于固定频率傅里叶方法,接近MLP性能
  • 理论证明有效,适合追求可解释性与高效逼近的模型研究者

Kolmogorov-Arnold表示定理指出,任意连续多变量函数均可精确表示为有限个单变量连续函数的叠加。后续简化将这些函数表示为较少唯一单调函数的参数化求和,由此推导出带S形激活函数的多层感知机具有通用逼近能力,成为现代神经网络的理论基础。最近提出的Kolmogorov-Arnold网络(KAN)以可学习的非线性激活直接作用于输入,通过基样条函数的加权和建模,取代传统感知机的线性变换与S形后激活。本文提出一种新型KAN变体,将原定理中的内层与外层函数均替换为可学习频率的加权正弦函数。受Lorentz和Sprecher简化解法启发,固定正弦激活相位为线性分布常数,并提供理论有效性证明。数值实验表明,该方法在多种多变量函数上优于固定频率傅里叶变换,性能接近多层感知机(MLPs)。

原文摘要 · Abstract (English)

The Kolmogorov-Arnold representation theorem states that any continuous multivariable function can be exactly represented as a finite superposition of continuous single variable functions. Subsequent simplifications of this representation involve expressing these functions as parameterized sums of a smaller number of unique monotonic functions. These developments led to the proof of the universal approximation capabilities of multilayer perceptron networks with sigmoidal activations, forming the alternative theoretical direction of most modern neural networks. Kolmogorov-Arnold Networks (KANs) have been recently proposed as an alternative to multilayer perceptrons. KANs feature learnable nonlinear activations applied directly to input values, modeled as weighted sums of basis spline functions. This approach replaces the linear transformations and sigmoidal post-activations used in traditional perceptrons. Subsequent works have explored alternatives to spline-based activations. In this work, we propose a novel KAN variant by replacing both the inner and outer functions in the Kolmogorov-Arnold representation with weighted sinusoidal functions of learnable frequencies. Inspired by simplifications introduced by Lorentz and Sprecher, we fix the phases of the sinusoidal activations to linearly spaced constant values and provide a proof of its theoretical validity. We also conduct numerical experiments to evaluate its performance on a range of multivariable functions, comparing it with fixed-frequency Fourier transform methods and multilayer perceptrons (MLPs). We show that it outperforms the fixed-frequency Fourier transform and achieves comparable performance to MLPs.

KAN正弦激活函数逼近神经网络理论

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。