多跳检索增强生成会放大语音识别错误，导致问答准确率大幅下降。
Better Retrieval, Worse Robustness: How Multi-hop RAG Amplifies Upstream ASR Errors
- 引入实体图链接与迭代重述，但加剧了上游语音识别误差
- 在高误识率语音下，性能下降幅度比基础检索大36%-67%
- 主要失败原因是查询实体被错误识别，适用于语音交互系统研究者
基于语音的应用在检索前需经过自动语音识别（ASR），ASR错误作为固定上游干扰进入流程。我们实证检验了两种标准检索增强生成（RAG）的扩展——实体图链接和迭代重述——是否能缓解或放大这些错误。使用四种通过神经文本转语音合成的英语口音，在三个多跳问答基准（HotpotQA、2WikiMultiHopQA、MuSiQue）上评估四种RAG配置，并与干净文本基线对比。尽管结构更丰富的配置在ASR输入下整体保持更高绝对F1，但两种扩展均放大错误：在所有三个基准上，从干净文本到最高误识率口音的F1差距，其组合比基础密集检索大36%-67%。主要失败模式为一个或多个查询实体被错误识别，占2WikiMultiHopQA上所有方法87%-96%的性能退化案例。两种轻量级表面形式缓解措施未能显著缩小差距，表明下游检索结构会放大残余实体错误。代码与数据已公开于https://github.com/Continuum-AI-Corp/spoken-multihop-rag。
原文摘要 · Abstract (English)
Speech-based applications pass spoken queries through automatic speech recognition (ASR) before any retrieval module, so ASR errors enter the pipeline as a fixed upstream constraint. We empirically test whether two extensions to standard retrieval-augmented generation (RAG), entity-graph linking and iterative reformulation, absorb or amplify these errors. Using four English accents synthesized through neural TTS, we evaluate four RAG configurations on three multi-hop QA benchmarks (HotpotQA, 2WikiMultiHopQA and MuSiQue) against a clean-text oracle. Although the structurally richer configurations generally retain higher absolute F1 under ASR input, both extensions amplify the error: the F1 gap from clean text to the highest-WER accent is 36-67% larger under their combination than under naive dense retrieval, on all three benchmarks. The dominant failure mode is corruption of one or more query entities, accounting for 87-96% of degradation cases on 2WikiMultiHopQA across all four methods. Two lightweight surface-form mitigations leave most of the gap intact, indicating that downstream retrieval structure amplifies remaining entity errors. We release code and data at https://github.com/Continuum-AI-Corp/spoken-multihop-rag .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。