针对电力客服复杂查询,融合意图识别与多路检索提升准确率
Enhancing Retrieval-Augmented Generation for Electric Power Industry Customer Support
- 通过意图拆解和多路检索融合,精准处理多意图复杂问题
- 在生成数据集上达97.9%准确率,在真实电网问答集上89.6%
- 剔除关键词增强,采用重排与图结构检索,减少幻觉
许多AI客服系统依赖标准NLP流程或微调语言模型,难以应对模糊、多意图或细节敏感的查询。本案例研究评估了查询重写、RAG Fusion、关键词增强、意图识别和上下文重排等技术,用于构建电力行业稳健的客户支持系统。比较向量存储与图结构RAG框架后,选择图结构RAG因其在复杂查询上的优越表现。查询重写提升了使用非标准术语或需精确细节查询的召回率;RAG Fusion通过融合多源检索结果,改善模糊或多面查询性能;重排机制通过过滤无关上下文降低幻觉;意图识别可将复杂问题分解为更精准子查询,提升相关性与效率。相反,关键词增强因关键词选择偏差而负面影响结果。最终系统结合意图识别、RAG Fusion与重排,有效处理歧义与多源查询。在GPT-4生成数据集与真实电力公司FAQ数据集上分别取得97.9%与89.6%准确率,显著优于基线RAG模型。
原文摘要 · Abstract (English)
Many AI customer service systems use standard NLP pipelines or finetuned language models, which often fall short on ambiguous, multi-intent, or detail-specific queries. This case study evaluates recent techniques: query rewriting, RAG Fusion, keyword augmentation, intent recognition, and context reranking, for building a robust customer support system in the electric power domain. We compare vector-store and graph-based RAG frameworks, ultimately selecting the graph-based RAG for its superior performance in handling complex queries. We find that query rewriting improves retrieval for queries using non-standard terminology or requiring precise detail. RAG Fusion boosts performance on vague or multifaceted queries by merging multiple retrievals. Reranking reduces hallucinations by filtering irrelevant contexts. Intent recognition supports the decomposition of complex questions into more targeted sub-queries, increasing both relevance and efficiency. In contrast, keyword augmentation negatively impacts results due to biased keyword selection. Our final system combines intent recognition, RAG Fusion, and reranking to handle disambiguation and multi-source queries. Evaluated on both a GPT-4-generated dataset and a real-world electricity provider FAQ dataset, it achieves 97.9% and 89.6% accuracy respectively, substantially outperforming baseline RAG models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。