让大模型学会在错误检索下仍不编造，提升事实准确性。
Finetune-RAG: Fine-Tuning Language Models to Resist Hallucination in Retrieval-Augmented Generation
- 用模拟真实检索缺陷的数据微调模型，增强抗幻觉能力。
- 相比基线模型，事实准确率提升21.2%。
- 适合需要高可靠性生成的医疗、法律等场景。
检索增强生成（RAG）通过将大语言模型输出与检索到的文档结合，提升了模型的事实性。然而，确保相关信息被正确检索仍具挑战性；当无关内容被传入模型时，可能导致幻觉。本文提出Finetune-RAG，一种简单有效的微调方法，其核心是构建首个模拟现实检索缺陷的RAG训练数据集。实验表明，Finetune-RAG相较基线模型事实准确率提升21.2%。我们还提出了Bench-RAG——一个基于大模型评判的评估流程，可在真实不完美检索条件下压力测试模型表现。代码与数据集已开源，供社区使用。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) has emerged as a powerful framework to improve factuality in large language models (LLMs) by grounding their outputs in retrieved documents. However, ensuring perfect retrieval of relevant information remains challenging, and when irrelevant content is passed downstream to an LLM, it can lead to hallucinations. In this work, we propose Finetune-RAG, a simple and effective fine-tuning approach that features the first-of-its-kind RAG training dataset constructed to mimic real-world imperfections. Experimental results show that Finetune-RAG improves factual accuracy by 21.2% over the base model. We also propose Bench-RAG, an LLM-as-a-judge evaluation pipeline that stress tests models under realistic imperfect retrieval scenarios. Our codebase and dataset are fully open sourced for community use.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。