RAGVUE可精准诊断生成错误来源，让评估结果透明可解释
RAGVUE: A Diagnostic View for Explainable and Automated Evaluation of Retrieval-Augmented Generation
- 将RAG流程分解为检索质量、回答相关性等四类可解释指标
- 在对比实验中发现传统工具忽略的细粒度错误
- 支持自动化评估与交互式调试，适合研发与研究场景
评估检索增强生成（RAG）系统仍具挑战：现有指标常将异构行为压缩为单一分数，且难以判断错误源自检索、推理还是事实一致性。本文提出RAGVUE，一个用于RAG流水线的诊断性、可解释的自动化、无参考评估框架。RAGVUE将RAG行为分解为检索质量、答案相关性与完整性、严格层面的事实一致性及判别校准四类指标，每项均提供结构化解释，使评估过程透明。该框架支持手动选择指标或全自动化代理评估，提供Python API、CLI和本地Streamlit界面，便于交互使用。在对比实验中，RAGVUE揭示了RAGAS等工具常忽略的细粒度失败。本文展示了完整RAGVUE工作流，并说明其如何融入研究与实际RAG开发流程。源代码与详细使用说明已公开于GitHub。
原文摘要 · Abstract (English)
Evaluating Retrieval-Augmented Generation (RAG) systems remains a challenging task: existing metrics often collapse heterogeneous behaviors into single scores and provide little insight into whether errors arise from retrieval,reasoning, or grounding. In this paper, we introduce RAGVUE, a diagnostic and explainable framework for automated, reference-free evaluation of RAG pipelines. RAGVUE decomposes RAG behavior into retrieval quality, answer relevance and completeness, strict claim-level faithfulness, and judge calibration. Each metric includes a structured explanation, making the evaluation process transparent. Our framework supports both manual metric selection and fully automated agentic evaluation. It also provides a Python API, CLI, and a local Streamlit interface for interactive usage. In comparative experiments, RAGVUE surfaces fine-grained failures that existing tools such as RAGAS often overlook. We showcase the full RAGVUE workflow and illustrate how it can be integrated into research pipelines and practical RAG development. The source code and detailed instructions on usage are publicly available on GitHub
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。