arXiv:2511.19324cs.IRcs.AI2025-11被引 2

多语言检索应优先用语义对齐模型,而非翻译。

What Drives Cross-lingual Ranking? Retrieval Approaches with Multilingual Language Models

  • 用预训练编码器做稠密检索,比词法匹配更有效。
  • 对比学习显著提升低资源和跨书写系统语言的检索效果。
  • 适合需要跨语言检索的低资源语言研究者使用。

跨语言信息检索(CLIR)使多语言知识获取成为可能,但受限于资源差异、文字体系不同以及嵌入模型中弱跨语言语义对齐。现有流程常依赖翻译与单语检索启发式方法,增加计算开销并引入噪声,降低性能。本文系统评估了四种干预方式:文档翻译、基于预训练编码器的多语言稠密检索、词、短语及查询-文档层级的对比学习,以及交叉编码器重排序,在三个基准数据集上的表现。结果表明,专为CLIR训练的稠密检索模型始终优于词法匹配方法,且从文档翻译中获益甚微。对比学习能缓解语言偏见,对初始对齐较弱的编码器带来显著提升;重排序虽有效,但依赖高质量交叉编码器训练数据。尽管高资源语言仍占优势,但在低资源和跨书写系统语言对上,相比词法和文档翻译基线的提升最为明显。这表明跨语言搜索系统应优先采用语义对齐的多语言嵌入和针对性学习对齐,尤其适用于跨书写系统和资源匮乏的语言。

原文摘要 · Abstract (English)

Cross-lingual information retrieval (CLIR) enables access to multilingual knowledge but remains challenging due to disparities in resources, scripts, and weak cross-lingual semantic alignment in embedding models. Existing pipelines often rely on translation and monolingual retrieval heuristics, which add computational overhead and noise, degrading performance. This work systematically evaluates four intervention types, namely document translation, multilingual dense retrieval with pretrained encoders, contrastive learning at word, phrase, and query-document levels, and cross-encoder re-ranking, across three benchmark datasets. We find that dense retrieval models trained specifically for CLIR consistently outperform lexical matching methods and derive little benefit from document translation. Contrastive learning mitigates language biases and yields substantial improvements for encoders with weak initial alignment, and re-ranking can be effective, but depends on the quality of the cross-encoder training data. Although high-resource languages still dominate overall performance, gains over lexical and document-translated baselines are most pronounced for low-resource and cross-script pairs. These findings indicate that cross-lingual search systems should prioritise semantic multilingual embeddings and targeted learning-based alignment over translation-based pipelines, particularly for cross-script and under-resourced languages.

跨语言检索稠密检索对比学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。