arXiv:2412.15540cs.CL2024-12被引 20

提出模块化检索框架MRAG,提升大模型对时序问题的回答能力

MRAG: A Modular Retrieval Framework for Time-Sensitive Question Answering

  • 将时序问题分解为内容与时间约束,分步处理
  • 在TempRAGEval上比基线检索器提升显著,答案准确率更高
  • 无需训练,适合需要快速响应的时序问答场景

理解时间关系并回答时序敏感问题是大型语言模型问答系统的关键挑战。现有方法要么通过更新参数知识来引入新事实,成本高且不切实际;要么结合外部知识检索(即检索增强生成)。然而,现成检索器常难以识别需复杂时间推理的相关文档。为此,我们构建了TempRAGEval基准,通过引入时间扰动和黄金证据标签重构现有数据集。结果表明,所有现有检索方法在该任务上表现不佳。我们进一步提出无需训练的模块化检索框架MRAG,包含三个模块:(1) 问题处理,将问题分解为主内容和时间约束;(2) 检索与摘要,检索证据并用大模型按主内容摘要;(3) 语义-时序混合排序,基于语义与时间相关性评分每条摘要。在TempRAGEval上,MRAG显著优于基线检索器,进而提升最终答案准确率。

原文摘要 · Abstract (English)

Understanding temporal relations and answering time-sensitive questions is crucial yet a challenging task for question-answering systems powered by large language models (LLMs). Existing approaches either update the parametric knowledge of LLMs with new facts, which is resource-intensive and often impractical, or integrate LLMs with external knowledge retrieval (i.e., retrieval-augmented generation). However, off-the-shelf retrievers often struggle to identify relevant documents that require intensive temporal reasoning. To systematically study time-sensitive question answering, we introduce the TempRAGEval benchmark, which repurposes existing datasets by incorporating temporal perturbations and gold evidence labels. As anticipated, all existing retrieval methods struggle with these temporal reasoning-intensive questions. We further propose Modular Retrieval (MRAG), a trainless framework that includes three modules: (1) Question Processing that decomposes question into a main content and a temporal constraint; (2) Retrieval and Summarization that retrieves evidence and uses LLMs to summarize according to the main content; (3) Semantic-Temporal Hybrid Ranking that scores each evidence summarization based on both semantic and temporal relevance. On TempRAGEval, MRAG significantly outperforms baseline retrievers in retrieval performance, leading to further improvements in final answer accuracy.

时序问答检索增强大模型无训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。