用7B小模型验证了多跳问答中简单检索更有效
Dissecting Agentic RAG: A Component Ablation for Multi-Hop QA with a Local 7B Model
- 固定混合检索优于自适应路由,提升1.8%准确率
- 两次检索迭代已捕获95%性能增益,深度循环无额外收益
- 查询分解和重排序带来显著提升,适合资源受限场景
代理式检索增强生成(Agentic RAG)系统通过迭代推理、查询分解和自适应检索来应对多跳问答任务。然而,在仅使用本地7B参数模型的资源受限环境下,各组件的贡献仍不清晰。本文在HotpotQA消歧开发集上对包含5000个问题的完整RAG流程进行受控消融实验,使用Qwen2.5-7B-Instruct模型。完整管道取得EM=53.2%、F1=61.6%,远超单次密集检索基线(EM=43.1%,F1=54.0%)。结果表明:(1)通过倒数排名融合的固定混合检索始终优于基于规则的自适应路由(+1.8 EM,+1.9 F1),因路由策略过度依赖命名实体触发BM25;(2)两次子问题检索已覆盖五次迭代95%的增益,深层循环无明显提升;(3)查询分解与交叉编码器重排序均带来统计显著提升(p<0.01 和 p<0.001)。在固定本地模型预算下,简单且固定的策略优于自适应设计,主要收益来自短周期检索而非复杂路由或多次迭代。全程未使用任何专有API或大规模算力。
原文摘要 · Abstract (English)
Agentic retrieval-augmented generation (RAG) systems combine iterative reasoning loops, query decomposition, and adaptive retrieval to tackle multi-hop question answering. However, the contribution of each component remains poorly understood, particularly under resource-constrained settings using only local language models. Many agentic designs add adaptive retrieval routing and deeper retrieval loops on the assumption that the added complexity helps. To test whether it does, we run a controlled ablation study of a full agentic RAG pipeline evaluated on 5,000 questions from the HotpotQA distractor development set using a local 7B parameter model (Qwen2.5-7B-Instruct). Our full pipeline achieves EM=53.2% and F1=61.6%, compared to a single-pass dense-retrieval baseline of EM=43.1% and F1=54.0%. Across eight ablation conditions, we find that: (1) fixed hybrid retrieval via reciprocal rank fusion consistently outperforms rule-based adaptive routing (+1.8 EM, +1.9 F1), as the routing heuristic over-routes to BM25 by firing on named entities present in nearly all multi-hop sub-questions; (2) two retrieval iterations over the decomposed sub-questions capture 95% of the gains of five, with no meaningful benefit from deeper loops; and (3) query decomposition and cross-encoder reranking each contribute statistically significant but smaller gains (p<0.01 and p<0.001 respectively). Taken together, on a fixed local-model budget, the simpler and fixed choices turn out to be competitive with or better than their adaptive versions: most of the gain comes from running a short retrieval loop, not from adaptive routing or from many iterations. We use no proprietary APIs or large-scale compute.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。