提升越南语法律问答系统可靠性，优化文档检索与排序
Vietnamese Legal Information Retrieval in Question-Answering System
- 改进越南语数据处理，减少嵌入模型的词元冗余
- 用归一化互斥排名融合法提升关键词与向量搜索效果
- 通过主动检索重排源文档，增强生成答案的准确性与用户体验
在数据量快速膨胀的现代背景下，精准检索与推荐相关文档对提升问答（QA）系统的可靠性至关重要。近年来，检索增强生成（RAG）通过缓解大语言模型（LLM）在问答中的幻觉问题，显著增强了系统能力，尤其适用于法律领域。已有方法如使用密集向量嵌入进行语义搜索，或结合多种技术在输入LLM前优化结果，但在越南语场景下常因越南语数据处理效率低导致词元过长，或集成策略过于简单引发结果不稳定、改进有限。此外，一个常被忽视的关键问题是最终相关文档的排序，直接影响LLM生成答案的准确性。本文提出三项核心改进：第一，探索多种实用的数据处理方法以克服嵌入模型局限；第二，通过归一化互斥排名融合（Reciprocal Rank Fusion）有效整合关键词与向量搜索结果；第三，采用主动检索（Active Retrieval）对LLM使用的源信息片段进行精细重排，显著提升用户在信息精炼过程中的体验。我们认为该方法可作为传统交叉编码器的替代重排策略。将这些技术整合进综合问答系统后，系统性能与可靠性均得到显著提升。
原文摘要 · Abstract (English)
In the modern era of rapidly increasing data volumes, accurately retrieving and recommending relevant documents has become crucial in enhancing the reliability of Question Answering (QA) systems. Recently, Retrieval Augmented Generation (RAG) has gained significant recognition for enhancing the capabilities of large language models (LLMs) by mitigating hallucination issues in QA systems, which is particularly beneficial in the legal domain. Various methods, such as semantic search using dense vector embeddings or a combination of multiple techniques to improve results before feeding them to LLMs, have been proposed. However, these methods often fall short when applied to the Vietnamese language due to several challenges, namely inefficient Vietnamese data processing leading to excessive token length or overly simplistic ensemble techniques that lead to instability and limited improvement. Moreover, a critical issue often overlooked is the ordering of final relevant documents which are used as reference to ensure the accuracy of the answers provided by LLMs. In this report, we introduce our three main modifications taken to address these challenges. First, we explore various practical approaches to data processing to overcome the limitations of the embedding model. Additionally, we enhance Reciprocal Rank Fusion by normalizing order to combine results from keyword and vector searches effectively. We also meticulously re-rank the source pieces of information used by LLMs with Active Retrieval to improve user experience when refining the information generated. In our opinion, this technique can also be considered as a new re-ranking method that might be used in place of the traditional cross encoder. Finally, we integrate these techniques into a comprehensive QA system, significantly improving its performance and reliability
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。