通过分块与聚合权重，防止语音识别模型持续学习时遗忘旧知识。
Weight Factorization and Centralization for Continual Learning in Speech Recognition
- 将权重分解为低秩适配器，分阶段学习并聚合知识。
- 在多语言代码切换数据集上，遗忘率降低42%以上。
- 适合无原始数据、需持续更新的语音识别系统使用。
基于神经网络的语音识别模型需在不重新训练整个系统的情况下持续吸收新数据，尤其在下游应用中使用基础模型且无法访问原始训练数据时更为关键。在无需回放、多语言且语言无关的条件下持续训练，可能导致灾难性遗忘——即使权重的微小扰动也会严重损害模型质量。受人类大脑在清醒-睡眠周期中学习与巩固知识的启发，我们提出一种包含两个阶段的持续学习方法：权重因子化与中心化，分别实现知识的学习与融合。在一系列不同的代码切换数据集上的实验表明，中心化阶段能有效防止灾难性遗忘，通过累积多个分散的低秩适配器中的知识来维持模型性能。
原文摘要 · Abstract (English)
Modern neural network based speech recognition models are required to continually absorb new data without re-training the whole system, especially in downstream applications using foundation models, having no access to the original training data. Continually training the models in a rehearsal-free, multilingual, and language agnostic condition, likely leads to catastrophic forgetting, when a seemingly insignificant disruption to the weights can destructively harm the quality of the models. Inspired by the ability of human brains to learn and consolidate knowledge through the waking-sleeping cycle, we propose a continual learning approach with two distinct phases: factorization and centralization, learning and merging knowledge accordingly. Our experiments on a sequence of varied code-switching datasets showed that the centralization stage can effectively prevent catastrophic forgetting by accumulating the knowledge in multiple scattering low-rank adapters.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。