通过添加文档摘要提升法律RAG系统的检索可靠性
Towards Reliable Retrieval in RAG Systems for Large Legal Datasets
- 用文档级摘要增强文本分块,补充丢失的全局信息
- 在多个法律数据集上显著降低错误文档检索率
- 无需专业领域知识,通用摘要策略效果更优
检索增强生成(RAG)是缓解大语言模型在法律应用中幻觉问题的有前景方法,但其可靠性高度依赖于检索步骤的准确性。在法律领域,大规模结构相似的文档数据库常导致检索系统失效。本文首次识别并量化了一种关键失败模式——文档级检索错配(DRM),即检索器从完全错误的源文档中提取信息。为缓解此问题,我们提出一种简单高效的改进方法:摘要增强分块(SAC),通过为每个文本块注入文档级合成摘要,恢复标准分块过程中丢失的关键全局上下文。在多种法律信息检索任务上的实验表明,SAC显著降低了DRM,同时提升了文本级检索的精确率与召回率。有趣的是,通用摘要策略的表现优于结合法律专家领域知识、针对特定法律要素的设计方案。本工作证明,该实用、可扩展且易于集成的技术能有效提升大规模法律文档数据集上RAG系统的可靠性。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) is a promising approach to mitigate hallucinations in Large Language Models (LLMs) for legal applications, but its reliability is critically dependent on the accuracy of the retrieval step. This is particularly challenging in the legal domain, where large databases of structurally similar documents often cause retrieval systems to fail. In this paper, we address this challenge by first identifying and quantifying a critical failure mode we term Document-Level Retrieval Mismatch (DRM), where the retriever selects information from entirely incorrect source documents. To mitigate DRM, we investigate a simple and computationally efficient technique which we refer to as Summary-Augmented Chunking (SAC). This method enhances each text chunk with a document-level synthetic summary, thereby injecting crucial global context that would otherwise be lost during a standard chunking process. Our experiments on a diverse set of legal information retrieval tasks show that SAC greatly reduces DRM and, consequently, also improves text-level retrieval precision and recall. Interestingly, we find that a generic summarization strategy outperforms an approach that incorporates legal expert domain knowledge to target specific legal elements. Our work provides evidence that this practical, scalable, and easily integrable technique enhances the reliability of RAG systems when applied to large-scale legal document datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。