让高维参数的KAN网络压缩80倍，精度不降。
MetaCluster: Enabling Deep Compression of Kolmogorov-Arnold Network
- 用轻量元学习器引导系数向量聚类，形成可压缩低维结构。
- 在图像和方程建模任务中实现最高80倍参数压缩，精度无损。
- 适合需要轻量化部署的深度学习模型研发者。
Kolmogorov-Arnold Networks（KANs）用每边的基函数系数向量替代标量权重，提升了表达能力和精度，但也导致参数和内存呈倍数增长。本文提出MetaCluster框架，使KAN在不损失精度的前提下高度可压缩。具体地，一个轻量级元学习器与KAN联合训练，将低维嵌入映射为系数向量，使其聚集在低维流形上，便于聚类。随后在系数空间运行K-means，将每边向量替换为共享质心。元学习器可丢弃，仅需对质心码本进行微调即可恢复精度损失。最终模型仅存储小规模码本和每边索引，利用系数向量特性实现多系数间的存储分摊。在MNIST、CIFAR-10和CIFAR-100上，使用多种基函数的标准KAN与ConvKAN均实现最高80倍参数存储压缩，精度无损。在高维方程建模任务中，参数压缩达124.1倍，性能不受影响。代码将在发表后公开。
原文摘要 · Abstract (English)
Kolmogorov-Arnold Networks (KANs) replace scalar weights with per-edge vectors of basis coefficients, thereby increasing expressivity and accuracy while also resulting in a multiplicative increase in parameters and memory. We propose MetaCluster, a framework that makes KANs highly compressible without sacrificing accuracy. Specifically, a lightweight meta-learner, trained jointly with the KAN, maps low-dimensional embeddings to coefficient vectors, thereby shaping them to lie on a low-dimensional manifold that is amenable to clustering. We then run K-means in coefficient space and replace per-edge vectors with shared centroids. Afterwards, the meta-learner can be discarded, and a brief fine-tuning of the centroid codebook recovers any residual accuracy loss. The resulting model stores only a small codebook and per-edge indices, exploiting the vector nature of KAN parameters to amortize storage across multiple coefficients. On MNIST, CIFAR-10, and CIFAR-100, across standard KANs and ConvKANs using multiple basis functions, MetaCluster achieves a reduction of up to $80\times$ in parameter storage, with no loss in accuracy. Similarly, on high-dimensional equation modeling tasks, MetaCluster achieves a parameter reduction of $124.1\times$, without impacting performance. Code will be released upon publication.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。