arXiv:2609.05281cs.SD2026-09

用新型神经网络模块让语音模型微调更高效且不丢性能。

KanAdapter: A Kolmogorov-Arnold Network-based Plug-and-Play Module for Efficient Fine-tuning of Foundation Speech Models

论文配图:KanAdapter: A Kolmogorov-Arnold Network-based Plug-and-Play Module for Efficient Fine-tuning of Foundation Speech Models
图 1 · 摘自论文原文
  • 用可学习的柯尔莫哥洛夫-阿诺德网络替代传统MLP,提升表达能力。
  • 参数减少97.5%仍保持高精度,连续学习下错误率降83.6%。
  • 适合资源有限场景,尤其擅长防止遗忘旧知识。

全量微调自监督语音模型计算开销巨大,现有参数高效方法多依赖固定激活函数的MLP适配器,限制了在小参数预算下的表达能力。本文提出KanAdapter,以分组有理柯尔莫哥洛夫-阿诺德网络(GR-KAN)模块取代传统MLP瓶颈,实现更优表达与效率。采用并行瓶颈结构,在冻结的Transformer编码器旁插入可训练的GR-KAN分支,并利用预训练MLP层权重进行稳定初始化。在说话人验证、语音情绪识别和深度伪造检测任务中,相比全量微调,参数减少高达97.5%,表现仍具竞争力,且在相同参数量下持续优于AdaptFormer。在持续学习场景中,错误率相较全量微调和MLP适配器降低达83.6%,归因于GR-KAN局部有理激活机制缓解灾难性遗忘。据我们所知,这是首个将KAN用于语音基础模型参数高效微调的工作。

原文摘要 · Abstract (English)

Fully fine-tuning self-supervised learning (SSL) speech models for downstream tasks is computationally prohibitive, and existing parameter-efficient fine-tuning approaches predominantly rely on MLP-based adapters whose fixed activation functions limit their representational expressiveness under tight parameter budgets. We propose \textbf{KanAdapter}, a lightweight adapter framework that replaces conventional MLP bottlenecks with Group-Rational Kolmogorov-Arnold Network (GR-KAN) modules for more expressive and parameter-efficient adaptation. Following a parallel bottleneck design, KanAdapter inserts trainable GR-KAN branches alongside frozen Transformer encoder blocks and leverages weight transfer from pre-trained MLP layers for stable initialization. Across speaker verification, speech emotion recognition, and deepfake detection, KanAdapter achieves up to 97.5\% reduction in trainable parameters relative to full fine-tuning while remaining highly competitive, and consistently outperforms AdaptFormer under comparable parameter budgets. In continual learning, it yields up to 83.6\% error reduction over full fine-tuning and MLP-based adapters, which we attribute to the localized nature of GR-KAN's rational activations that mitigates catastrophic forgetting. To our knowledge, this is the first work to explore KAN-based modules for parameter-efficient fine-tuning of speech foundation models.

语音模型高效微调KAN持续学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。