用多语言大模型生成扩展查询，提升跨语言检索效果
Generative Query Expansion with Multilingual LLMs for Cross-Lingual Information Retrieval
- 用多语言大模型生成伪文档来扩展查询
- 查询长度决定提示策略有效性，复杂提示未必更好
- 对低资源语言提升明显，但不同文字系统间仍存差距
查询扩展通过添加语义相关词重新表述用户查询，是单语和跨语言信息检索中避免遗漏相关文档的关键技术。近年来，多语言大语言模型（mLLMs）将查询扩展从同义词和相关词的语义增强，转向伪文档生成。伪文档不仅引入额外相关术语，还能弥合短查询与长文档之间的差距，尤其在稠密检索中优势显著。本研究评估了多种mLLMs及其微调变体在不同生成式扩展策略下的表现，发现查询长度很大程度上决定了提示技术的有效性，更复杂的提示并不总带来性能提升。语言间差异依然显著：跨语言查询扩展对基线最弱的语言改善最大，但在不同书写系统之间检索性能尤其低下。微调仅在训练与测试数据格式相似时带来性能提升。这些结果凸显了需要更均衡的多语言与跨语言训练及评估资源。
原文摘要 · Abstract (English)
Query expansion is the reformulation of a user query by adding semantically related information, and is an essential component of monolingual and cross-lingual information retrieval used to ensure that relevant documents are not missed. Recently, multilingual large language models (mLLMs) have shifted query expansion from semantic augmentation with synonyms and related words to pseudo-document generation. Pseudo-documents both introduce additional relevant terms and bridge the gap between short queries and long documents, which is particularly beneficial in dense retrieval. This study evaluates recent mLLMs and fine-tuned variants across several generative expansion strategies to identify factors that drive cross-lingual retrieval performance. Results show that query length largely determines which prompting technique is effective, and that more elaborate prompts often do not yield further gains. Substantial linguistic disparities persist: cross-lingual query expansion can produce the largest improvements for languages with the weakest baselines, yet retrieval is especially poor between languages written in different scripts. Fine-tuning is found to lead to performance gains only when the training and test data are of similar format. These outcomes underline the need for more balanced multilingual and cross-lingual training and evaluation resources.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。