用反向训练增强多语言语义对齐，提升低资源语言检索效果
CLEAR: Cross-Lingual Enhancement in Alignment via Reverse-training
- 通过英语文本作为桥梁，反向训练强化目标语言与英语的对齐
- 在低资源语言上最高提升15%，英文性能几乎不受影响
- 适合多语言检索、低资源语言处理场景，易于集成到现有模型
现有多语言嵌入模型在跨语言场景中常因语言资源不均衡且训练时忽视跨语言对齐而表现不佳。尽管对比学习广泛用于跨语言适配，但仍难以捕捉深层语言对齐，甚至导致英语等高资源语言性能下降。为此，我们提出跨语言检索对齐增强方法CLEAR，采用反向训练机制，利用英语段落作为桥梁，强化目标语言与英语之间的对齐，从而提升跨语言检索性能。大量实验表明，CLEAR在跨语言场景中实现显著提升，尤其在低资源语言上最高达15%的增益，同时几乎不降低英语表现。此外，该方法在多语言训练中也表现出色，具备良好可扩展性。代码已开源：https://github.com/dltmddbs100/CLEAR。
原文摘要 · Abstract (English)
Existing multilingual embedding models often encounter challenges in cross-lingual scenarios due to imbalanced linguistic resources and less consideration of cross-lingual alignment during training. Although standardized contrastive learning approaches for cross-lingual adaptation are widely adopted, they may struggle to capture fundamental alignment between languages and degrade performance in well-aligned languages such as English. To address these challenges, we propose Cross-Lingual Enhancement in Retrieval via Reverse-training (CLEAR), a novel loss function utilizing a reverse training scheme to improve retrieval performance across diverse cross-lingual retrieval scenarios. CLEAR leverages an English passage as a bridge to strengthen alignments between the target language and English, ensuring robust performance in the cross-lingual retrieval task. Our extensive experiments demonstrate that CLEAR achieves notable improvements in cross-lingual scenarios, with gains up to 15%, particularly in low-resource languages, while minimizing performance degradation in English. Furthermore, our findings highlight that CLEAR offers promising effectiveness even in multilingual training, suggesting its potential for broad application and scalability. We release the code at https://github.com/dltmddbs100/CLEAR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。