提出多语言多跳问答新框架,解决跨语言信息检索难题
DaPT: A Dual-Path Framework for Multilingual Multi-hop Question Answering
- 并行生成源语言与英文翻译的子问题图,再融合统一处理
- 在最挑战的MuSiQue数据集上相对基线提升18.3%准确率
- 首个支持多语言多跳问答的系统,适合跨语言知识推理场景
检索增强生成(RAG)系统在英语多跳问答任务中取得显著进展,但在跨语言语料与查询的场景下面临挑战。现有研究缺乏评估多语言多跳(MM-hop)问答能力的基准,且过度依赖大模型在英语中的语义理解能力,在多语言环境下效果下降。为此,我们首先将仅含英语的基准数据集翻译为五种语言,构建多语言多跳问答基准;随后提出DaPT框架,该框架并行生成源语言查询及其英文翻译的子问题图,合并后采用双语检索-回答策略逐级求解子问题。实验表明,先进RAG系统在多语言场景中表现严重失衡。而所提方法在所有基准上均更准确、更简洁,显著提升性能:例如在最困难的MuSiQue数据集上,平均精确匹配(EM)分数相较最强基线提升18.3%。
原文摘要 · Abstract (English)
Retrieval-augmented generation (RAG) systems have made significant progress in solving complex multi-hop question answering (QA) tasks in the English scenario. However, RAG systems inevitably face the application scenario of retrieving across multilingual corpora and queries, leaving several open challenges. The first one involves the absence of benchmarks that assess RAG systems' capabilities under the multilingual multi-hop (MM-hop) QA setting. The second centers on the overreliance on LLMs' strong semantic understanding in English, which diminishes effectiveness in multilingual scenarios. To address these challenges, we first construct multilingual multi-hop QA benchmarks by translating English-only benchmarks into five languages, and then we propose DaPT, a novel multilingual RAG framework. DaPT generates sub-question graphs in parallel for both the source-language query and its English translation counterpart, then merges them before employing a bilingual retrieval-and-answer strategy to sequentially solve sub-questions. Our experimental results demonstrate that advanced RAG systems suffer from a significant performance imbalance in multilingual scenarios. Furthermore, our proposed method consistently yields more accurate and concise answers compared to the baselines, significantly enhancing RAG performance on this task. For instance, on the most challenging MuSiQue benchmark, DaPT achieves a relative improvement of 18.3\% in average EM score over the strongest baseline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。