无需标注数据，自动微调RAG模型提升跨语言问答准确率
ALoFTRAG: Automatic Local Fine Tuning for Retrieval Augmented Generation
- 自动生成并筛选合成训练数据，结合LoRA微调提升模型性能
- 在20个数据集、26种语言上平均提升引用准确率8.3%、回答准确率3.0%
- 适用于医疗、金融等敏感领域，低成本且保护数据隐私
检索增强生成（RAG）系统被证明可提升大语言模型输出的准确性。然而，在新数据领域中，这些模型常表现不佳。我们提出自动局部微调检索增强生成模型框架（ALoFTRAG），在不依赖人工标注数据或更大教师模型的前提下，提升RAG在特定领域的准确性。通过生成并过滤合成训练数据，结合LoRA微调，ALoFTRAG在20个数据集、26种语言上平均提升引用准确率8.3%、回答准确率3.0%。结果表明，ALoFTRAG提供了一种实用、低成本且数据安全的RAG性能优化方案，尤其适用于医疗、金融等敏感领域。
原文摘要 · Abstract (English)
Retrieval Augmented Generation (RAG) systems have been shown to improve the accuracy of Large Language Model (LLM) outputs. However, these models can often achieve low accuracy when applied to new data domains. We introduce the Automatic Local Fine Tuning of Retrieval Augmented Generation models (ALoFTRAG) framework, designed to improve the accuracy of RAG systems on a given domain by training LLMs without manually labeled data or using larger teacher models. By generating and filtering synthetic training data and performing LoRA fine-tuning, ALoFTRAG improves citation and answer accuracy across 20 datasets in 26 languages by, on average, 8.3% and 3.0% respectively. Our results demonstrate that ALoFTRAG offers a practical, cost-effective, and data-secure solution for improving RAG accuracy, making it particularly applicable to sensitive domains such as healthcare and finance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。