arXiv:2602.05235cs.CL2026-02被引 2

联邦检索增强生成新框架,隐私保护下提升问答准确率。

FedMosaic: Federated Retrieval-Augmented Generation via Parametric Adapters

  • 用可参数化的适配器替代原文传输,避免泄露数据
  • 通过文档聚类与掩码机制降低存储通信开销80%以上
  • 仅聚合相关且不冲突的适配器,提升生成准确性10.9%

检索增强生成(RAG)通过引入外部知识提升大语言模型的事实性并减少幻觉。然而,现有方法多依赖集中式知识库,在隐私敏感领域难以应用。为此提出联邦RAG(FedRAG),在不共享原始文档的前提下实现多方协作。传统上下文RAG需传输原文,而参数化RAG将文档编码为轻量适配器,与冻结的LLM在推理时融合。本文采用参数化方法,但面临两个挑战:单文档适配器导致高存储与通信开销,以及盲目合并适配器造成信息破坏。提出首个基于参数适配器的联邦RAG框架FedMosaic,通过语义聚类将相关文档整合为多文档适配器,并引入文档级掩码以保留特异性;同时实施选择性适配器聚合,仅融合语义对齐且无冲突的适配器。实验表明,FedMosaic在四个类别任务中平均准确率比现有最优方法高10.9%,存储成本降低78.8%至86.3%,通信成本降低91.4%,且始终不共享原始文档。

原文摘要 · Abstract (English)

Retrieval-Augmented Generation (RAG) enhances Large Language Models (LLMs) by grounding generation in external knowledge to improve factuality and reduce hallucinations. Yet most deployments assume a centralized corpus, which is infeasible in privacy aware domains where knowledge remains siloed. This motivates federated RAG (FedRAG), where a central LLM server collaborates with distributed silos without sharing raw documents. In context RAG violates this requirement by transmitting verbatim documents, whereas parametric RAG encodes documents into lightweight adapters that merge with a frozen LLM at inference, avoiding raw-text exchange. We adopt the parametric approach but face two unique challenges induced by FedRAG: high storage and communication from per-document adapters, and destructive aggregation caused by indiscriminately merging multiple adapters. We present FedMosaic, the first federated RAG framework built on parametric adapters. FedMosaic clusters semantically related documents into multi-document adapters with document-specific masks to reduce overhead while preserving specificity, and performs selective adapter aggregation to combine only relevance-aligned, nonconflicting adapters. Experiments show that FedMosaic achieves an average 10.9% higher accuracy than state-of-the-art methods in four categories, while lowering storage costs by 78.8% to 86.3% and communication costs by 91.4%, and never sharing raw documents.

联邦学习检索增强大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。