用奇异值调参，仅存一语音片段就能高效防遗忘。
Efficient Rehearsal for Continual Learning in ASR via Singular Value Tuning
- 通过奇异值分解控制参数更新，只重训少量门控向量。
- 每任务仅用1条语音片段，仍显著减少遗忘率。
- 适合资源受限或隐私敏感的语音识别持续学习场景。
自动语音识别（ASR）中的持续学习面临灾难性遗忘问题，尤其在适应新任务、新领域或新说话人时。现有基于回放的方法需存储大量历史数据，但存储成本高、难以适配预训练模型，且受隐私法规限制。本文提出一种新型回放式持续学习方法，在极小内存下仍保持高效：先在新任务上微调，再对线性层权重变化进行奇异值分解，仅以参数高效方式重新训练控制更新幅度的门控向量。我们在两个单语和两个多语基准上全面测试该方法，结果表明，即使每旧任务仅保留一条语音片段，仍能显著降低遗忘并超越当前最优的ASR持续学习方法。
原文摘要 · Abstract (English)
Continual Learning (CL) in Automatic Speech Recognition (ASR) suffers from catastrophic forgetting when adapting to new tasks, domains, or speakers. A common strategy to mitigate this is to store a subset of past data in memory for rehearsal. However, rehearsal-based methods face key limitations: storing data is often costly, infeasible with pre-trained models, or restricted by privacy regulations. Running existing rehearsal-based methods with smaller memory sizes to alleviate these issues usually leads to degraded performance. We propose a rehearsal-based CL method that remains effective even with minimal memory. It operates in two stages: first, fine-tuning on the new task; second, applying Singular Value Decomposition (SVD) to the changes in linear layers and, in a parameter-efficient manner, retraining only gating vectors on the singular values, which control to extent to which updates from the first stage are accepted, using rehearsal. We extensively test and analyze our method on two monolingual and two multilingual benchmarks. Our method reduces forgetting and outperforms state-of-the-art CL approaches for ASR, even when limited to a single utterance per previous task.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。