用可学习样条构建高效模型，适合资源受限设备部署。
Sprecher Networks: A Parameter-Efficient Kolmogorov-Arnold Architecture
- 基于柯尔莫戈洛夫-阿诺德表示，每块仅用两个共享样条实现多变量函数。
- 参数量随宽度线性增长，比传统MLP低阶，支持超宽浅层结构。
- 实现在4MB内存设备上实时分类，适合边缘计算场景。
我们提出Sprecher网络（SNs），一种基于1965年David Sprecher构造性形式的柯尔莫戈洛夫-阿诺德表示的可训练架构。每个SN模块通过两个共享可学习样条（单调内层ϕ和通用外层Φ）、可学习偏移参数η及跨输出维度共享的混合向量λ，实现“平移单变量函数之和”。堆叠这些模块形成深层组合模型；对向量输出，额外添加非求和输出块。我们还引入可选的横向混合算子，以仅增加$O(d_{\mathrm{out}})$参数的方式实现输出通道间通信。由于采用向量而非矩阵混合权重并共享样条，SNs在宽度上呈线性扩展，总参数量约$O(\sum_{\ell}(d_{\ell-1}+d_{\ell}+G))$（G为样条节点数），远低于密集MLP的$O(\sum_{\ell} d_{\ell-1}d_{\ell})$和边样条KAN的$O(G\sum_{\ell} d_{\ell-1}d_{\ell})$。这一线性扩展对极宽浅层模型尤为有利，可降低推理延迟。此外，我们提出一种顺序前向实现，避免生成$ d_{\mathrm{in}} \times d_{\mathrm{out}} $的中间张量，将峰值中间内存从二次降至线性，适用于内存受限场景，如嵌入式设备。我们在监督回归、Fashion-MNIST分类（含25层残差与归一化稳定训练）及泊松物理信息神经网络（Poisson PINN）上进行了实验，并与MLP和KAN基线进行对比验证。
原文摘要 · Abstract (English)
We introduce Sprecher Networks (SNs), a family of trainable architectures derived from David Sprecher's 1965 constructive form of the Kolmogorov-Arnold representation. Each SN block implements a "sum of shifted univariate functions" using only two shared learnable splines per block, a monotone inner spline $ϕ$ and a general outer spline $Φ$, together with a learnable shift parameter $η$ and a mixing vector $λ$ shared across all output dimensions. Stacking these blocks yields deep, compositional models; for vector-valued outputs we append an additional non-summed output block. We also propose an optional lateral mixing operator enabling intra-block communication between output channels with only $O(d_{\mathrm{out}})$ additional parameters. Owing to the vector (not matrix) mixing weights and spline sharing, SNs scale linearly in width, approximately $O(\sum_{\ell}(d_{\ell-1}+d_{\ell}+G))$ parameters for $G$ spline knots, versus $O(\sum_{\ell} d_{\ell-1}d_{\ell})$ for dense MLPs and $O(G\sum_{\ell} d_{\ell-1}d_{\ell})$ for edge-spline KANs. This linear width-scaling is particularly attractive for extremely wide, shallow models, where low depth can translate into low inference latency. Finally, we describe a sequential forward implementation that avoids materializing the $d_{\mathrm{in}}\times d_{\mathrm{out}}$ shifted-input tensor, reducing peak forward-intermediate memory from quadratic to linear in layer width, relevant for memory-constrained settings such as on-device/edge inference; we demonstrate deployability via fixed-point real-time digit classification on resource-constrained embedded device with only 4 MB RAM. We provide empirical demonstrations on supervised regression, Fashion-MNIST classification (including stable training at 25 hidden layers with residual connections and normalization), and a Poisson PINN, with controlled comparisons to MLP and KAN baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。