跨语言检索在阿拉伯语-英语场景下表现差,因文档排序困难导致生成失败。
The Cross-Lingual Cost: Retrieval Biases in RAG over Arabic-English Corpora
- 构建真实企业数据集上的多语言检索测试基准,覆盖所有语言组合。
- 当查询与文档语言不同时,性能显著下降,最高降幅达30%以上。
- 通过等量检索或查询翻译策略,显著提升跨语言检索效果,适合实际应用。
跨语言检索增强生成(RAG)在多语言问答中至关重要。以往研究多基于开放域数据集如Wikipedia,但其语言平衡性、预训练数据重叠及记忆内容掩盖了真实检索挑战。本文在领域特定的企业数据集上构建了覆盖用户查询与支持文档所有语言组合的测试基准,实现对多语言检索行为的系统分析。结果表明,在跨语言领域场景中,检索是主要瓶颈,当查询与文档语言不一致时性能显著下降。关键发现是:检索失败源于跨语言文档排序困难。为此提出两种简单策略——强制双语言等量检索或翻译查询,显著提升跨语言及整体性能。研究揭示了改善多语言检索的实际机会,尤其适用于现实世界中的RAG应用。
原文摘要 · Abstract (English)
Cross-lingual retrieval-augmented generation (RAG) is a critical capability for retrieving and generating answers across languages. Prior work in this context has mostly focused on generation and relied on benchmarks derived from open-domain sources, most notably Wikipedia. In such settings, retrieval challenges often remain hidden due to language imbalances, overlap with pretraining data, and memorized content. To address this gap, we study Arabic-English RAG in a domain-specific setting using benchmarks derived from real-world corporate datasets. Our benchmarks include all combinations of languages for the user query and the supporting document, drawn independently and uniformly at random. This enables a systematic study of multilingual retrieval behavior. Our findings reveal that retrieval is a critical bottleneck in cross-lingual domain-specific scenarios, with substantial performance drops occurring when the user query and supporting document languages differ. A key insight is that these failures stem primarily from the retriever's difficulty in ranking documents across languages. Finally, we propose two simple retrieval strategies that address this source of failure by enforcing equal retrieval from both languages or by translating the query, resulting in substantial improvements in cross-lingual and overall performance. These results highlight meaningful opportunities for improving multilingual retrieval, particularly in practical, real-world RAG applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。