arXiv:2608.00585cs.CLcs.IR2026-08

多跳问答中逐段验证无效,分解问题后验证效果显著提升

Verification Without Sufficiency: Per-Chunk Filtering Fails on Multi-Hop RAG, and Decomposition Repairs It

  • 用子问题替代原问题进行验证,突破逐段评分局限
  • 在MuSiQue上验证准确率从0.546提升至0.840,提升0.355
  • 无需额外训练,可用现成模型实现,适合多跳问答系统优化

检索增强生成中的验证通常对每个检索段落打分并剔除不合格项。我们发现这对多跳问题无效,因为多跳问题中任一单段都不是回答的充分前提,答案所在段落恰恰是问题未提及的。在HotpotQA、2WikiMultihopQA和MuSiQue上,蕴含打分的AUC分别为0.643、0.523和0.560,远低于单跳SQuAD的0.951。七组对照实验排除了模型容量、前提长度、假设模板、决策阈值、检索器、答案匹配标准和提示的影响。端到端测试中,无论数据集、生成器规模或提示方式,逐段过滤均显著劣于不过滤,且随生成器能力增强而恶化。修复方法是将验证条件改为分解后的子问题。使用MuSiQue的黄金分解,后续步骤的蕴含分数从0.546(随机水平)升至0.840,配对提升+0.355(置信区间[0.331, 0.382])。一个现成的Qwen2.5-7B分解器在给定问题和最高检索段落后,达到0.637,捕获了该上限的31%;无检索的分解仅达0.533,低于原始问题。迭代检索系统本已生成此类分解却在验证前丢弃。

原文摘要 · Abstract (English)

Verification for retrieval-augmented generation usually scores each retrieved chunk and drops the ones that fail. We show this cannot work for multi-hop questions, and show what does. Per-chunk scoring assumes one chunk is a sufficient premise for the answer. Multi-hop questions are built so that none is, and the paragraph carrying the answer is the one the question does not name. Entailment scoring reaches 0.643, 0.523 and 0.560 AUC on HotpotQA, 2WikiMultihopQA and MuSiQue, against 0.951 on single-hop SQuAD. Seven controls rule out model capacity, premise length, hypothesis template, decision threshold, retriever, answer-matching criterion and prompt. End to end across three datasets, three generator sizes and two prompts, per-chunk gating is significantly worse than not filtering at all in every cell, and its penalty grows with generator capability. The repair is to condition verification on the decomposed sub-question rather than the original query. Using MuSiQue's gold decomposition, entailment on a later hop rises from 0.546, which is chance, to 0.840, a paired lift of +0.355 with a bootstrap interval of [0.331, 0.382]. An off-the-shelf Qwen2.5-7B decomposer, given the question and the top retrieved paragraph, reaches 0.637 and captures 31% of that ceiling; decomposing without retrieval reaches 0.533, below the original question. Iterative retrieval systems already produce such decompositions and discard them before verifying.

多跳问答检索增强验证机制子问题分解

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。