发现检索重写中答案出现才是性能提升主因,而非证据质量改善。
Answer Presence Drives RAG Rewriting Gains
- 通过控制编辑重写文本中的答案位置,验证答案存在与否对结果的影响。
- 移除答案使模型性能下降28至64点,插入答案可提升0.7至9.7点。
- 揭示现有评估方法易受提示词干扰,适合验证其他重写模型效果。
检索增强问答系统常将检索段落经大模型重写后输入小模型阅读器,多跳任务上F1提升数十点,通常归因于证据质量提升。本文通过受控干预审计,检验该提升是否由黄金答案出现在重写内容中导致。对每个重写结果进行四种操作:移除答案、替换为等长非答案片段(安慰剂)、在缺失处前置或插入答案。在12组(单元,基线)实验中,移除答案导致F1下降28至64点,显著高于安慰剂;在10/12组合中,插入答案可使F1提升0.7至9.7点。另开展五哨兵审计,发现传统单[掩码]探针本身脆弱,在2WikiMultihopQA上报告+4.12的“无泄露残差”,换用四类替代哨兵后转为-3.33至-7.81,且三组未通过等价性检验(仅1/4通过)。本文不提出新重写器或缓解方案,而是发布干预运行器与哨兵面板,供他人验证重写增益的真实性。
原文摘要 · Abstract (English)
Retrieval-augmented QA pipelines often route retrieved passages through an LLM \emph{rewriter} before a smaller reader, lifting F1 by tens of points on multi-hop benchmarks; this gain is typically credited to improved evidence quality. We ask whether that lift is causally driven by the gold answer string appearing in the rewritten context rather than by curation per se, using a controlled intervention audit. For each rewritten context we re-run the reader after one of four controlled edits to the compile output: removing the gold answer span, replacing a length-matched random non-answer span (placebo), or injecting the gold into rewrites where it was absent (at the prefix or at a midpoint sentence boundary). Across twelve completed (cell, baseline) intervention runs spanning three reader families (Qwen2.5-7B, Qwen3.5-35B, GLM-4.7), two datasets (HotpotQA, 2WikiMultihopQA), and three compiler arrangements (MA-only, MB-only, MA$+$verify), removing the gold answer drops reader F1 by $28$ to $64$ points beyond the length-matched placebo on paired \texttt{answer-in-compile} strata, and prepending the gold into rewrites that lacked it raises F1 by $+0.7$ to $+9.7$ points in $10$ of $12$ (cell, baseline) combinations. A companion five-sentinel audit shows the conventional single-\texttt{[MASK]} probe is itself sentinel-fragile: on 2Wiki it reports a $+4.12$~F1 ``non-leakage residual'' that flips to $-3.33$ to $-7.81$~F1 under four alternative sentinels and fails an equivalence test for three of those four ($1/4$~pass). We do not propose a new rewriter or mitigation; we release the intervention runner and the sentinel panel so that other rewriter-gain claims can be tested against the same standard.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。