用树形结构共享编码，实现多语言翻译高效并行。
Transformer-Encoder Trees for Efficient Multilingual Machine Translation and Speech Translation
- 构建层次化编码树，共享相似语言中间表示。
- 参数减少66%，推理计算降低60%，支持单次前向传播生成所有语言。
- 适合需要快速多语言翻译的场景，尤其提升低资源语言质量。
多语言翻译存在计算冗余问题，尤其在同时翻译多种语言时；此外，低资源语言的翻译质量也往往较差。为此，我们提出Transformer Encoder Tree（TET），一种分层的非自回归编码器架构，采用连接时序分类（CTC）进行训练。TET在语义相近的目标语言间共享中间表示，从而提升低资源语言的准确率，同时减少计算冗余，并可在一次前向传播中生成所有目标语言。TET消除了自回归模型的序列瓶颈，支持所有目标语言所有词元的完全并行解码。与朴素的一对多多语言设计相比，TET将总参数量减少66%，推理计算降低60%。在语音翻译中，结合非自回归语音识别骨干（Wav2Vec2）时，TET在翻译质量上可媲美自回归系统，同时推理速度提升约7-14倍。
原文摘要 · Abstract (English)
Multilingual translation suffers from computational redundancy, especially when translating into multiple languages simultaneously. In addition, translation quality can suffer for low-resource languages. To address this, we introduce Transformer Encoder Tree (TET), a hierarchical, non-autoregressive encoder-only architecture trained with Connectionist Temporal Classification (CTC) for multilingual translation. TET shares intermediate representations among linguistically similar target languages, improving accuracy on low-resource languages while reducing computational redundancy and enabling the generation of all target languages in a single forward pass. TET eliminates the sequential bottleneck of autoregressive models and supports fully parallel decoding of all tokens across all target languages. Compared to a naive one-to-many multilingual design, TET reduces the total parameter count by 66% and lowers inference computation by 60%. In speech translation, combining TET with a non-autoregressive speech recognition backbone (Wav2Vec2) shows competitive translation quality compared to autoregressive systems while speeding up inference by approximately 7-14 times.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。