用倒排问题匹配提升问答准确率,解决大模型幻觉问题
QuIM-RAG: Advancing Retrieval-Augmented Generation with Inverted Question Matching for Enhanced QA Performance
- 将文档切片生成潜在问题,通过倒排索引匹配用户提问
- 在500+页自建语料上,比传统RAG的BERT-Score和RAGAS均更高
- 适合需要高精度问答的场景,如智能客服、知识库系统
本文提出一种新型RAG架构QuIM-RAG,用于提升从目标语料中进行问答任务的性能。大型语言模型虽能生成类人文本,但依赖预训练数据,缺乏实时更新能力。RAG通过整合在线资源弥补此缺陷,但面临信息稀释和幻觉等问题。本方法将语料转换为领域专用数据集,并构建响应生成系统。引入新检索机制QuIM-RAG(问题到问题倒排索引匹配),从文档片段生成潜在问题,与用户查询匹配以定位最相关文本块。我们在Meta开源的LLaMA3-8B-instruct模型基础上实现该系统,使用来自日访问量高的网站的500+页自建语料,及人工标注的真实问答对进行评估。采用BERT-Score与RAGAS两项先进指标对比传统RAG模型,结果表明本方法在两项指标上均表现更优。
原文摘要 · Abstract (English)
This work presents a novel architecture for building Retrieval-Augmented Generation (RAG) systems to improve Question Answering (QA) tasks from a target corpus. Large Language Models (LLMs) have revolutionized the analyzing and generation of human-like text. These models rely on pre-trained data and lack real-time updates unless integrated with live data tools. RAG enhances LLMs by integrating online resources and databases to generate contextually appropriate responses. However, traditional RAG still encounters challenges like information dilution and hallucinations when handling vast amounts of data. Our approach addresses these challenges by converting corpora into a domain-specific dataset and RAG architecture is constructed to generate responses from the target document. We introduce QuIM-RAG (Question-to-question Inverted Index Matching), a novel approach for the retrieval mechanism in our system. This strategy generates potential questions from document chunks and matches these with user queries to identify the most relevant text chunks for generating accurate answers. We have implemented our RAG system on top of the open-source Meta-LLaMA3-8B-instruct model by Meta Inc. that is available on Hugging Face. We constructed a custom corpus of 500+ pages from a high-traffic website accessed thousands of times daily for answering complex questions, along with manually prepared ground truth QA for evaluation. We compared our approach with traditional RAG models using BERT-Score and RAGAS, state-of-the-art metrics for evaluating LLM applications. Our evaluation demonstrates that our approach outperforms traditional RAG architectures on both metrics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。