MetaRAG让大模型学会自我评估检索，提升复杂问答准确率。
A Reproducibility Study of Metacognitive Retrieval-Augmented Generation
- 引入元认知机制，让模型自主判断何时停止检索
- 实测准确率低于原文，但优于标准RAG和推理基线
- 重排器显著提升性能，对额外特征更鲁棒
近期，检索增强生成（RAG）转向多步检索以应对多跳问答等复杂任务。然而，系统难以判断何时停止检索。为此，周等人提出元认知检索增强生成（MetaRAG），受元认知启发，使大模型能自我批评与修正推理。本文复现该框架并拓展两点：(i) 评估点式与列表式重排器效果，(ii) 与采用轻量级批评模型的SIM-RAG进行对比。结果确认MetaRAG相较标准RAG和基于推理的基线有相对提升，但绝对分数低于原文报告，反映出闭源大模型更新、实现细节缺失及未公开提示词带来的挑战。我们表明MetaRAG部分可复现，重排显著增益，且在扩展检索特征后比SIM-RAG更具鲁棒性。
原文摘要 · Abstract (English)
Recently, Retrieval Augmented Generation (RAG) has shifted focus to multi-retrieval approaches to tackle complex tasks such as multi-hop question answering. However, these systems struggle to decide when to stop searching once enough information has been gathered. To address this, \citet{zhou2024metacognitive} introduced Metacognitive Retrieval Augmented Generation (MetaRAG), a framework inspired by metacognition that enables Large Language Models to critique and refine their reasoning. In this reproducibility paper, we reproduce MetaRAG following its original experimental setup and extend it in two directions: (i) by evaluating the effect of PointWise and ListWise rerankers, and (ii) by comparing with SIM-RAG, which employs a lightweight critic model to stop retrieval. Our results confirm MetaRAG's relative improvements over standard RAG and reasoning-based baselines, but also reveal lower absolute scores than reported, reflecting challenges with closed-source LLM updates, missing implementation details, and unreleased prompts. We show that MetaRAG is partially reproduced, gains substantially from reranking, and is more robust than SIM-RAG when extended with additional retrieval features.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。