混合检索生成技术提升法规文本查准查全率
A Hybrid Approach to Information Retrieval and Answer Generation for Regulatory Texts
- 融合词法与语义搜索，结合BM25与微调句向量模型
- 召回率@10和MAP@10显著优于单一方法
- 适合合规审查、法律智能辅助等专业场景
法规文本通常冗长复杂，给信息检索系统带来挑战。本文提出一种混合式信息检索系统，结合词法与语义搜索技术，从大规模法规语料中提取相关资讯。系统将微调的句子变换器模型与传统BM25算法集成，兼顾语义精准性与词法覆盖范围。为生成准确完整的回答，通过大语言模型在检索增强生成（RAG）框架下整合检索到的段落。实验表明，该混合系统显著优于独立的词法与语义方法，在Recall@10和MAP@10上均有明显提升。我们开源了微调模型与方法，旨在推动监管领域合规应用的自然语言处理工具发展。
原文摘要 · Abstract (English)
Regulatory texts are inherently long and complex, presenting significant challenges for information retrieval systems in supporting regulatory officers with compliance tasks. This paper introduces a hybrid information retrieval system that combines lexical and semantic search techniques to extract relevant information from large regulatory corpora. The system integrates a fine-tuned sentence transformer model with the traditional BM25 algorithm to achieve both semantic precision and lexical coverage. To generate accurate and comprehensive responses, retrieved passages are synthesized using Large Language Models (LLMs) within a Retrieval Augmented Generation (RAG) framework. Experimental results demonstrate that the hybrid system significantly outperforms standalone lexical and semantic approaches, with notable improvements in Recall@10 and MAP@10. By openly sharing our fine-tuned model and methodology, we aim to advance the development of robust natural language processing tools for compliance-driven applications in regulatory domains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。