对比三种RAG方法，发现图谱增强版在机场对话中更准更安全。
Evaluating the Performance of RAG Methods for Conversational AI in the Airport Domain
- 用三种RAG架构处理机场动态问答，其中图谱RAG结合知识图谱提升推理能力。
- 图谱RAG准确率达91.49%，远超传统RAG的84.84%且极少幻觉。
- 适合对安全要求高的场景，如机场调度、术语解析和复杂推理任务。
为提升机场自动化水平，我们构建了一个面向机场工作人员的对话式AI系统，可与航班信息系统交互，回答标准问题并解析术语、缩写及需推理的动态问题。本文设计了三种检索增强生成（RAG）方法：传统RAG、SQL RAG与基于知识图谱的RAG（Graph RAG）。实验表明，传统RAG在使用BM25 + GPT-4时达到84.84%准确率，但存在明显幻觉，影响安全；而SQL RAG与Graph RAG分别实现80.85%与91.49%准确率，幻觉显著减少。尤其在涉及推理的问题上，Graph RAG表现突出。因此建议在机场环境中优先采用SQL RAG与Graph RAG，因其更可靠且能应对动态信息需求。
原文摘要 · Abstract (English)
Airports from the top 20 in terms of annual passengers are highly dynamic environments with thousands of flights daily, and they aim to increase the degree of automation. To contribute to this, we implemented a Conversational AI system that enables staff in an airport to communicate with flight information systems. This system not only answers standard airport queries but also resolves airport terminology, jargon, abbreviations, and dynamic questions involving reasoning. In this paper, we built three different Retrieval-Augmented Generation (RAG) methods, including traditional RAG, SQL RAG, and Knowledge Graph-based RAG (Graph RAG). Experiments showed that traditional RAG achieved 84.84% accuracy using BM25 + GPT-4 but occasionally produced hallucinations, which is risky to airport safety. In contrast, SQL RAG and Graph RAG achieved 80.85% and 91.49% accuracy respectively, with significantly fewer hallucinations. Moreover, Graph RAG was especially effective for questions that involved reasoning. Based on our observations, we thus recommend SQL RAG and Graph RAG are better for airport environments, due to fewer hallucinations and the ability to handle dynamic questions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。