FedRAG让RAG系统在联邦学习下也能高效微调。
FedRAG: A Framework for Fine-Tuning Retrieval-Augmented Generation Systems
- 支持集中式与联邦式训练的RAG微调框架
- 兼容主流微调方法，可平滑切换训练模式
- 深度集成现代RAG生态，填补工具空白
检索增强生成（RAG）系统已被证明能有效缓解大语言模型仅依赖参数记忆的缺陷。近期研究显示，通过微调检索器和生成器模型可进一步提升RAG性能。本文提出FedRAG，一个支持集中式与联邦架构的RAG系统微调框架。该框架兼容当前最先进的微调方法，提供简洁直观的接口，并支持从集中式到联邦训练的无缝转换。此外，FedRAG深度集成于现代RAG生态系统，弥补了现有工具链的关键缺口。
原文摘要 · Abstract (English)
Retrieval-augmented generation (RAG) systems have been shown to be effective in addressing many of the drawbacks of relying solely on the parametric memory of large language models. Recent work has demonstrated that RAG systems can be improved via fine-tuning of their retriever and generator models. In this work, we introduce FedRAG, a framework for fine-tuning RAG systems across centralized and federated architectures. FedRAG supports state-of-the-art fine-tuning methods, offering a simple and intuitive interface and a seamless conversion from centralized to federated training tasks. FedRAG is also deeply integrated with the modern RAG ecosystem, filling a critical gap in available tools.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。