SimulMEGA让专家路由自动决策语音翻译的读写时机，兼顾速度与质量。
SimulMEGA: MoE Routers are Advanced Policy Makers for Simultaneous Speech Translation
- 用专家混合机制隐式学习读写策略，无需额外推理开销。
- 在6种语言对上，1.5秒延迟下BLEU损失低于7%，3秒时低于3%。
- 适用于多语种实时语音翻译与语音合成，通用性强。
同时性语音翻译（SimulST）通过在严格延迟约束下联合优化语音识别与机器翻译，实现跨语言实时通信。现有系统难以平衡翻译质量、延迟与语义连贯性，尤其在多语言多对多场景中，读写策略差异阻碍统一策略学习。本文提出SimulMEGA（基于专家混合门控的同时生成），一种无监督策略学习框架，结合前缀训练与专家混合精炼器，以隐式方式学习有效的读写决策，且不增加推理开销。该设计仅需对标准Transformer架构进行最小修改，可泛化至语音到文本及文本到语音的流式任务。在六种语言对上的综合评估表明，500M参数的语音到文本模型优于Seamless基线，在平均1.5秒延迟下BLEU下降不足7%，3秒延迟下不足3%。进一步实验显示，将SimulMEGA扩展至单向骨干的流式语音合成任务，亦能取得更优的延迟-质量权衡。
原文摘要 · Abstract (English)
Simultaneous Speech Translation (SimulST) enables real-time cross-lingual communication by jointly optimizing speech recognition and machine translation under strict latency constraints. Existing systems struggle to balance translation quality, latency, and semantic coherence, particularly in multilingual many-to-many scenarios where divergent read and write policies hinder unified strategy learning. In this paper, we present SimulMEGA (Simultaneous Generation by Mixture-of-Experts Gating), an unsupervised policy learning framework that combines prefix-based training with a Mixture-of-Experts refiner to learn effective read and write decisions in an implicit manner, without adding inference-time overhead. Our design requires only minimal modifications to standard transformer architectures and generalizes across both speech-to-text and text-to-speech streaming tasks. Through comprehensive evaluation on six language pairs, our 500M parameter speech-to-text model outperforms the Seamless baseline, achieving under 7 percent BLEU degradation at 1.5 seconds average lag and under 3 percent at 3 seconds. We further demonstrate the versatility of SimulMEGA by extending it to streaming TTS with a unidirectional backbone, yielding superior latency quality tradeoffs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。