开源复现CRAG并分析其纠错机制，性能相当且可解释。
Open-Source Reproduction and Explainability Analysis of Corrective Retrieval Augmented Generation
- 用Wikipedia API和Phi-3-mini替代原版闭源组件，实现完全开源。
- 在PopQA和ARC-Challenge上性能接近原系统，准确率分别为78.2%和64.5%。
- 首次用SHAP分析检索评估器，发现其依赖实体对齐而非语义相似性。
纠错型检索增强生成（CRAG）通过评估检索文档质量并触发纠正动作，提升了RAG系统的鲁棒性。但原实现依赖谷歌搜索API和闭源模型权重，限制了可复现性。本文提出完全开源的CRAG复现方案，将原版网络搜索替换为Wikipedia API，将原始LLaMA-2生成器替换为Phi-3-mini-4k-instruct。在PopQA和ARC-Challenge数据集上评估显示，开源管道性能与原系统相当（分别为78.2%和64.5%准确率）。此外，我们首次对基于T5的检索评估器进行可解释性分析，使用SHAP发现其主要依赖命名实体对齐而非语义相似性。分析揭示关键失败模式，如科学类问题在跨领域迁移中的表现局限。所有代码与结果已公开于https://github.com/suryayalavarthi/crag-reproduction。
原文摘要 · Abstract (English)
Corrective Retrieval Augmented Generation (CRAG) improves the robustness of RAG systems by evaluating retrieved document quality and triggering corrective actions. However, the original implementation relies on proprietary components including the Google Search API and closed model weights, limiting reproducibility. In this work, we present a fully open-source reproduction of CRAG, replacing proprietary web search with the Wikipedia API and the original LLaMA-2 generator with Phi-3-mini-4k-instruct. We evaluate on PopQA and ARC-Challenge, demonstrating that our open-source pipeline achieves comparable performance to the original system. Furthermore, we contribute the first explainability analysis of CRAG's T5-based retrieval evaluator using SHAP, revealing that the evaluator primarily relies on named entity alignment rather than semantic similarity. Our analysis identifies key failure modes including domain transfer limitations on science questions. All code and results are available at https://github.com/suryayalavarthi/crag-reproduction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。