用AI打通科研设施的海量隐性知识，让工作人员能自然语言提问并获得精准答案。
A corrective agentic hybrid RAG and an operations-grounded evaluation for a scientific facility
- 融合稠密、稀疏和知识图谱三路检索，用自适应融合策略提升召回率。
- 引入纠错式智能体循环，最终关键信息召回率达70.3%，比基线高6.5个百分点。
- 专为科研设施设计评估体系，适合想部署AI助手的大型科学仪器团队。
科研用户设施积累了数十年的操作经验，涵盖电子日志、技术文档、内部维基、运维聊天记录、维护档案及实时控制系统数据，但无单一索引可覆盖全部内容。本文提出APS-RAG——先进光子源检索增强生成平台，使阿贡国家实验室先进光子源（APS）的机构知识可通过自然语言查询获取，并构建了基于实际运维的评估体系。检索引擎融合稠密、稀疏与知识图谱（KG）三通道，采用查询类型自适应的倒数排名融合策略；引入纠错式智能体循环，并在模型上下文协议（MCP）工具层上运行原生工具的ReAct执行器。我们构建了包含50个问题的APS-Bench评测集，每个问题均有可审计的正确答案。所有增强检索方案均显著优于基础的BM25基线（63.8%），其中完整纠错式智能体图谱RAG达到70.3%的严格关键信息召回率。交叉编码器重排序对答案质量贡献显著：若移除该模块并由大模型自行判断相关性，严格关键召回率骤降32.8%。图谱通道与纠错循环虽有正向作用，但提升有限。此外，还对比了开源与闭源大模型在最终答案生成中的表现。我们公开了APS-Bench构建方法、六层评估框架、底层代码库及'/aps-rag'检索代理技能框架，以支持其他设施复现与应用。整体平台与基于运维的评估体系，为大型科学仪器的可信、统计可验证的AI辅助操作提供可迁移的工作流。
原文摘要 · Abstract (English)
Scientific user facilities accumulate decades of operational knowledge that no single search index covers: electronic logbooks, technical documents, internal wikis, operations chat messages, maintenance records, and live control-system data. We present APS-RAG, Advanced Photon Source Retrieval Augmented Generation, a deployed platform that makes the institutional knowledge at the Advanced Photon Source (APS) accessible to staff through natural-language queries, along with an operations-grounded evaluation. The retrieval engine fuses dense, sparse, and knowledge-graph (KG) channels with query-type-adaptive reciprocal-rank fusion, adds a corrective agentic loop, and runs a native-tool ReAct executor over a Model Context Protocol (MCP) tooling layer. We construct APS-Bench, a 50-question, question-answering (QA) dataset with auditable gold answers. Every retrieval-augmented variant numerically improves strict vital-nugget recall over a naive BM25 baseline (63.8%), with the full corrective Agentic GraphRAG scoring (70.3%). The cross-encoder reranker contributes significantly to answer quality: removing it and allowing the LLM to score relevance drastically reduces strict vital recall by 32.8%. The graph channel and corrective loop contribute positively as expected, but the performance gains are marginal. Additionally, we also compare the performance of open-source and closed-source LLMs in final answer synthesis. We release the APS-Bench construction methodology, the six-layer evaluation harness, and the underlying codebase, along with the '/aps-rag' retrieval agent skill framework, to support reproduction and adoption at other facilities. Together, the deployed platform and its operations-grounded evaluation present a promising workflow for trustworthy, statistically grounded AI assistance in facility operations, transferable to other large scientific instruments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。