融合多种方法打造跨语言句子嵌入模型，大幅降低对平行语料需求。
Lingua Custodi's participation at the WMT 2025 Terminology shared task
- 整合MLM、TLM等技术，构建多语言句子嵌入方法。
- 仅需原数据量20%的平行语料即达83.7%跨语言检索准确率。
- 适用于多语言信息检索与机器翻译任务，支持109+语言。
尽管BERT在单语句嵌入方面表现优异，但其跨语言句嵌入仍待探索。本文系统研究通过结合单语和跨语言表示学习的最佳方法——掩码语言建模（MLM）、翻译语言建模（TLM）、双编码器翻译排序及加性间隔Softmax——来构建多语言句嵌入。实验表明，引入预训练多语言模型可使达到良好性能所需的平行数据量减少80%。所提方法在包含112种语言的Tatoeba数据集上实现83.7%的双语文本检索准确率，显著优于LASER的65.5%。同时在单语迁移学习基准上保持竞争力。利用该模型从CommonCrawl挖掘的平行数据，可训练出媲美主流水平的英中、英德神经机器翻译模型。最佳模型已公开发布于https://tfhub.dev/google/LaBSE，支持109+语言。
原文摘要 · Abstract (English)
While BERT is an effective method for learning monolingual sentence embeddings for semantic similarity and embedding based transfer learning BERT based cross-lingual sentence embeddings have yet to be explored. We systematically investigate methods for learning multilingual sentence embeddings by combining the best methods for learning monolingual and cross-lingual representations including: masked language modeling (MLM), translation language modeling (TLM), dual encoder translation ranking, and additive margin softmax. We show that introducing a pre-trained multilingual language model dramatically reduces the amount of parallel training data required to achieve good performance by 80%. Composing the best of these methods produces a model that achieves 83.7% bi-text retrieval accuracy over 112 languages on Tatoeba, well above the 65.5 achieved by LASER, while still performing competitively on monolingual transfer learning benchmarks. Parallel data mined from CommonCrawl using our best model is shown to train competitive NMT models for en-zh and en-de. We publicly release our best multilingual sentence embedding model for 109+ languages at https://tfhub.dev/google/LaBSE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。