VERA通过多步评估与优化,提升检索增强语言模型的准确性和可靠性。
VERA: Validation and Enhancement for Retrieval Augmented systems
- 引入评估-增强双角色大模型,先验判断是否需外部检索并净化上下文。
- 对生成回答进行原子语句拆分,逐条验证其与查询的相关性与一致性。
- 适用于对准确性要求高的场景,尤其提升小模型表现,适合实际部署。
大型语言模型(LLMs)虽能力强大,但常因依赖内部知识而产生错误。检索增强生成(RAG)通过引入外部信息检索系统,在查询时提供额外上下文以减少特定情境下的不准确。然而,仍存在模型依赖无关文档或错误推断训练知识的问题。为此,我们提出VERA(Validation and Enhancement for Retrieval Augmented systems),一个用于评估与优化检索增强系统中检索内容和生成响应的框架:1)在生成前评估并增强检索到的上下文,剔除冗余信息;2)在生成后分析响应中的原子语句,确保其与查询相关且符合上下文。实验表明,VERA不仅显著提升了小型开源模型的表现,也增强了先进大模型的性能。其多步骤评估与优化机制有效缓解幻觉,提升检索与生成质量,为高精度、高可靠性的信息生成应用提供了有力工具。
原文摘要 · Abstract (English)
Large language models (LLMs) exhibit remarkable capabilities but often produce inaccurate responses, as they rely solely on their embedded knowledge. Retrieval-Augmented Generation (RAG) enhances LLMs by incorporating an external information retrieval system, supplying additional context along with the query to mitigate inaccuracies for a particular context. However, accuracy issues still remain, as the model may rely on irrelevant documents or extrapolate incorrectly from its training knowledge. To assess and improve the performance of both the retrieval system and the LLM in a RAG framework, we propose \textbf{VERA} (\textbf{V}alidation and \textbf{E}nhancement for \textbf{R}etrieval \textbf{A}ugmented systems), a system designed to: 1) Evaluate and enhance the retrieved context before response generation, and 2) Evaluate and refine the LLM-generated response to ensure precision and minimize errors. VERA employs an evaluator-cum-enhancer LLM that first checks if external retrieval is necessary, evaluates the relevance and redundancy of the retrieved context, and refines it to eliminate non-essential information. Post-response generation, VERA splits the response into atomic statements, assesses their relevance to the query, and ensures adherence to the context. Our experiments demonstrate VERA's remarkable efficacy not only in improving the performance of smaller open-source models, but also larger state-of-the art models. These enhancements underscore VERA's potential to produce accurate and relevant responses, advancing the state-of-the-art in retrieval-augmented language modeling. VERA's robust methodology, combining multiple evaluation and refinement steps, effectively mitigates hallucinations and improves retrieval and response processes, making it a valuable tool for applications demanding high accuracy and reliability in information generation. .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。