用引导令牌实现专家路由，提升多任务语音转文本模型性能
Beyond Hard Sharing: Efficient Multi-Task Speech-to-Text Modeling with Supervised Mixture of Experts
- 用特殊引导令牌代替门控网络，按任务分配专用专家
- 在编码器和解码器同时应用，使语音识别与翻译联合优化
- 在混合带宽输入下，词错误率降低6.35%，适合多任务语音模型
硬参数共享是联合训练多任务模型的常见策略，但常导致任务干扰，影响整体性能。为此，我们提出一种简单有效的监督型专家混合模型（S-MoE）。不同于传统MoE，S-MoE通过特殊引导令牌直接将各任务路由至指定专家，无需训练门控函数。每个任务由独立前馈网络处理，克服了硬参数共享的局限性。我们将S-MoE应用于语音转文本模型，在处理混合带宽输入的同时，联合执行自动语音识别（ASR）与语音翻译（ST）。实验表明，当在编码器和解码器中同时使用S-MoE时，词错误率（WER）相对降低6.35%。
原文摘要 · Abstract (English)
Hard-parameter sharing is a common strategy to train a single model jointly across diverse tasks. However, this often leads to task interference, impeding overall model performance. To address the issue, we propose a simple yet effective Supervised Mixture of Experts (S-MoE). Unlike traditional Mixture of Experts models, S-MoE eliminates the need for training gating functions by utilizing special guiding tokens to route each task to its designated expert. By assigning each task to a separate feedforward network, S-MoE overcomes the limitations of hard-parameter sharing. We further apply S-MoE to a speech-to-text model, enabling the model to process mixed-bandwidth input while jointly performing automatic speech recognition (ASR) and speech translation (ST). Experimental results demonstrate the effectiveness of the proposed S-MoE, achieving a 6.35% relative improvement in Word Error Rate (WER) when applied to both the encoder and decoder.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。