用结构化数据增强的RAG系统,让大模型更准地回答技术文档中的复杂问题。
LLM-Assisted Question-Answering on Technical Documents Using Structured Data-Aware Retrieval Augmented Generation
- 结合向量搜索与Gemma微调重排序器,精准定位文档中的表格和图像信息。
- 在表格类问题上表现优异,答案忠实度达94%~96%,相关性高达93%。
- 适合需要高精度回答技术文档中复杂问题的研究者或工程师使用。
大语言模型虽具备自然语言理解与生成能力,但存在幻觉和知识过时等问题。微调虽可缓解,但成本高且需随数据更新重复训练。检索增强生成(RAG)通过引入外部知识源提供高效解决方案,但传统RAG难以处理含表格、图像等结构化数据的技术文档。本文提出一种新型RAG管道,支持扫描版与可搜索版技术文档,其检索过程融合向量相似性搜索与基于Gemma-2-9b-it微调的重排序器。重排序器在自定义数据集上通过RAFT(检索增强微调)训练,以提升问答上下文识别能力。评估显示,该方案在答案忠实度上达到94%(RAGas)和96%(DeepEval),相关性达87%(RAGas)和93%(DeepEval)。对比分析表明,该架构在处理表格类问题及上下文外问题时优于通用RAG管道。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are capable of natural language understanding and generation. But they face challenges such as hallucination and outdated knowledge. Fine-tuning is one possible solution, but it is resource-intensive and must be repeated with every data update. Retrieval-Augmented Generation (RAG) offers an efficient solution by allowing LLMs to access external knowledge sources. However, traditional RAG pipelines struggle with retrieving information from complex technical documents with structured data such as tables and images. In this work, we propose a RAG pipeline, capable of handling tables and images in documents, for technical documents that support both scanned and searchable formats. Its retrieval process combines vector similarity search with a fine-tuned reranker based on Gemma-2-9b-it. The reranker is trained using RAFT (Retrieval-Augmented Fine-Tuning) on a custom dataset designed to improve context identification for question answering. Our evaluation demonstrates that the proposed pipeline achieves a high faithfulness score of 94% (RAGas) and 96% (DeepEval), and an answer relevancy score of 87% (RAGas) and 93% (DeepEval). Comparative analysis demonstrates that the proposed architecture is superior to general RAG pipelines in terms of table-based questions and handling questions outside context.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。