让神经网络的基函数可学习,提升模型表达能力。
RecKAN: Kolmogorov-Arnold Networks with a Learnable Recursive Polynomial Basis

- 用可学习的递推多项式定义基函数,替代固定形式的基。
- 在图像、文本等任务上优于三种基准KAN模型,预测误差更低。
- 递推系数可解释,能动态适应数据局部结构需求。
Kolmogorov-Arnold网络(KANs)将传统网络中固定的标量权重替换为每条边上的可学习一元函数,但现有方法仍固定这些函数所基于的基(如B样条、切比雪夫多项式等),仅学习组合权重。本文提出RecKAN,通过一个二阶多项式递推关系 $R_{n+1}(x) = (ax^2+bx+c)R_n(x) + (dx+e)R_{n-1}(x)$ 自适应定义基函数,其五个系数与网络一同学习。该递推可恢复切比雪夫多项式、斐波那契、佩尔、雅各布斯塔尔等经典多项式族作为特例,并证明其在特定子族中度数线性增长,赋予学习基超越固定基的能力。在图像、文本、生物医学时间序列分类和时间序列预测等多个基准上,RecKAN在所有分类任务中均优于三种参数匹配的KAN基线(切比雪夫、雅可比、样条),并在ETTh1预测任务上达到最低均方误差。作为卷积骨干的分类头,在Fashion MNIST、CIFAR-10和SVHN上表现优于标准MLP头。在合成函数拟合任务中,能逼近高度振荡的目标函数,而参数相当的MLP难以捕捉。进一步分析表明,学习到的递推系数具有可解释性:在要求局部结构最强的任务中,训练使基函数脱离线性度增长区域,与理论分析一致。
原文摘要 · Abstract (English)
Kolmogorov--Arnold Networks (KANs) replace the fixed scalar weights of a standard network with learnable univariate functions on each edge, but existing variants still fix the \emph{basis} that those functions are built from: B-splines, Chebyshev polynomials, wavelets, or Jacobi polynomials, and learn only the combination weights over it. We introduce RecKAN, which instead defines the basis itself by a second order polynomial recurrence, $R_{n+1}(x) = (ax^2+bx+c)R_n(x) + (dx+e)R_{n-1}(x)$, whose five coefficients are learned jointly with the network. We show this recurrence recovers several classical polynomial families including both kinds of Chebyshev polynomials, Fibonacci, Pell, and Jacobsthal polynomials as special cases, and prove that its degree grows linearly in $n$ exactly on the sub-family containing all of them, giving a concrete sense in which the learned basis can move beyond any fixed classical choice. Across multiple benchmark datasets spanning image, text, biomedical time series classification, and time series forecasting, RecKAN outperforms three parameter-matched KAN baselines (Chebyshev, Jacobi, and spline based) on all classification tasks and achieves the lowest MSE on the ETTh1 forecasting benchmark. Additionally, when used as a classifier head with a convolutional backbone, RecKAN achieves higher accuracy than standard MLP heads on Fashion MNIST, CIFAR-10, and SVHN. On a synthetic function fitting benchmark it tracks a sharply oscillatory target that a parameter comparable MLP under fits. We further show that the learned recurrence coefficients are interpretable: on the task requiring the most local structure, training moves the basis away from the linear degree growth regime that contains every classical family we identify, consistent with our theoretical analysis of what that structural shift enables.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。