轻量级框架让语音模型持续学新语言，不丢旧能力。
MiLorE-SSL: Scaling Multilingual Capabilities in Self-Supervised Models without Forgetting
- 用低秩适配+软专家混合，高效融合新语言
- 仅2.14%参数可训练，新旧语言性能都提升
- 用少量历史数据回放，避免遗忘且省资源
自监督学习（SSL）极大推动了语音表征的发展，但多语言SSL模型仍受限于预训练时接触的语言。从头重训成本高，顺序训练易引发灾难性遗忘。为此，我们提出MiLorE-SSL——一种轻量级框架，结合LoRA模块与软混合专家（MoE）机制，实现高效的持续多语言训练。LoRA提供低秩高效适配，软MoE促进跨语言专家共享，降低交叉干扰。为缓解遗忘，引入少量已有语言的回放数据，无需依赖大规模历史语料。在ML-SUPERB上的实验表明，MiLorE-SSL在新增语言上表现优异，同时提升了原有语言的能力，仅需2.14%的可训练参数。
原文摘要 · Abstract (English)
Self-supervised learning (SSL) has greatly advanced speech representation learning, but multilingual SSL models remain constrained to languages encountered during pretraining. Retraining from scratch to incorporate new languages is computationally expensive, while sequential training without migitation strategies often leads to catastrophic forgetting. To address this, we propose MiLorE-SSL, a lightweight framework that combines LoRA modules with a soft mixture-of-experts (MoE) mechanism for efficient continual multilingual training. LoRA provides efficient low-rank adaptation, while soft MoE promotes flexible expert sharing across languages, reducing cross-lingual interference. To further mitigate forgetting, we introduce limited replay data from existing languages, avoiding reliance on large historical corpora. Experiments on ML-SUPERB demonstrate that MiLorE-SSL achieves strong performance in new languages and improves the ability in existing ones with only 2.14% trainable parameters.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。