用可学习的样条网络替代线性探测，提升迁移学习效果
Reimagining Linear Probing: Kolmogorov-Arnold Networks in Transfer Learning
- 用样条函数构建KAN替代传统线性层，捕捉复杂数据关系
- 在ImageNet预训练的ResNet-50上，对CIFAR-10测试，准确率显著提升
- 通过网格大小和样条阶数调优，实现更强泛化能力，适合模型优化者
本文提出将柯尔莫戈洛夫-阿诺德网络（Kolmogorov-Arnold Networks, KAN）引入迁移学习中的线性探测方法。传统线性探测仅在预训练模型末尾使用线性层，难以建模复杂数据关系。为此，我们以样条函数为基础，用KAN替代线性探测层，以逼近复杂函数。研究中将KAN与在ImageNet上预训练的ResNet-50结合，在CIFAR-10数据集上进行系统评估，通过网格尺寸与样条阶数（k）的超参数搜索，优化其灵活性与精度。结果表明，KAN在多种配置下均显著优于传统线性探测，大幅提升了准确率与泛化性能。这证明了KAN是迁移学习中更强大、更灵活的替代方案。
原文摘要 · Abstract (English)
This paper introduces Kolmogorov-Arnold Networks (KAN) as an enhancement to the traditional linear probing method in transfer learning. Linear probing, often applied to the final layer of pre-trained models, is limited by its inability to model complex relationships in data. To address this, we propose substituting the linear probing layer with KAN, which leverages spline-based representations to approximate intricate functions. In this study, we integrate KAN with a ResNet-50 model pre-trained on ImageNet and evaluate its performance on the CIFAR-10 dataset. We perform a systematic hyperparameter search, focusing on grid size and spline degree (k), to optimize KAN's flexibility and accuracy. Our results demonstrate that KAN consistently outperforms traditional linear probing, achieving significant improvements in accuracy and generalization across a range of configurations. These findings indicate that KAN offers a more powerful and adaptable alternative to conventional linear probing techniques in transfer learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。