提出SSVD方法，用最少参数实现语音模型在不同语境下的鲁棒微调。
SSVD: Structured SVD for Parameter-Efficient Fine-Tuning and Benchmarking under Domain Shift in ASR
- 基于结构化SVD，只旋转输入相关的右奇异向量，保留输出映射稳定。
- 在0.1B到2B模型上测试，对儿童语音和方言变化均有显著提升。
- 首次在ESPnet中系统评测多类高效微调方法，代码开源可复现。
参数高效微调（PEFT）已成为适配大模型的可扩展方案。尽管低秩适应（LoRA）广泛应用于语音任务，但其顶尖变体如VeRA、DoRA、PiSSA和SVFT主要针对语言与视觉任务设计，且在语音领域验证有限。本文首次在ESPnet中系统集成并基准测试这些PEFT方法。我们进一步提出结构化SVD引导（SSVD）微调：仅选择性地旋转与输入相关的右奇异向量，同时固定输出相关向量以保持语义映射。该设计在极少可训练参数下实现稳健的域适应，显著提升效率。我们在多个跨域语音识别任务上评估所有方法，涵盖儿童语音与方言变异，覆盖0.1B至2B规模模型。所有实现均已发布于ESPnet，以支持可复现性与后续研究。
原文摘要 · Abstract (English)
Parameter-efficient fine-tuning (PEFT) has emerged as a scalable solution for adapting large foundation models. While low-rank adaptation (LoRA) is widely used in speech applications, its state-of-the-art variants, e.g., VeRA, DoRA, PiSSA, and SVFT, are developed mainly for language and vision tasks, with limited validation in speech. This work presents the first comprehensive integration and benchmarking of these PEFT methods within ESPnet. We further introduce structured SVD-guided (SSVD) fine-tuning, which selectively rotates input-associated right singular vectors while keeping output-associated vectors fixed to preserve semantic mappings. This design enables robust domain adaptation with minimal trainable parameters and improved efficiency. We evaluate all methods on domain-shifted speech recognition tasks, including child speech and dialectal variation, across model scales from 0.1B to 2B. All implementations are released in ESPnet to support reproducibility and future work.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。