首个双语混合查询基准，揭示多语言检索模型在跨语言搜索中的表现差异。
MiLQ: Benchmarking IR Models for Bilingual Web Search with Mixed Language Queries
- 构建首个真实双语混合查询测试集，评估多语言检索模型表现。
- 模型在混合查询上表现中等且不稳定，英语混合查询更利于匹配英文文档。
- 适合研究跨语言信息检索、双语用户行为及代码切换训练的学者。
尽管双语用户在网页搜索中常使用混合语言查询，但针对此类查询的信息检索研究仍很匮乏。为此，我们提出了MiLQ——首个公开的混合语言查询测试集，具备真实性和较高可接受度。实验表明，多语言信息检索模型在MiLQ上的表现中等，且在母语、英语和混合语言查询间表现不一致；同时，有意识地在查询中混用英语被证明是双语用户检索英文文档的有效策略，分析显示其优势源于相比母语查询更强的词元匹配能力。该结果提示代码切换训练数据可能有助于提升模型处理混合语言查询的鲁棒性。
原文摘要 · Abstract (English)
Despite bilingual speakers frequently using mixed-language queries in web searches, Information Retrieval (IR) research on them remains scarce. To address this, we introduce MiLQ, Mixed-Language Query test set, the first public benchmark of mixed-language queries, qualified as realistic and relatively preferred. Experiments show that multilingual IR models perform moderately on MiLQ and inconsistently across native, English, and mixed-language queries, also suggesting code-switched training data's potential for robust IR models handling such queries. Meanwhile, intentional English mixing in queries proves an effective strategy for bilinguals searching English documents, which our analysis attributes to enhanced token matching compared to native queries.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。