去中心化RAG系统通过区块链动态评估数据源可靠性,提升生成质量并降低成本。
A Decentralized Retrieval Augmented Generation System with Source Reliabilities Secured on Blockchain
- 用区块链智能合约动态评分数据源可靠性,优先调用高质量数据。
- 在不可靠数据环境下性能比中心化系统高10.7%,接近理想环境下的最优表现。
- 适合关注数据隐私、可信度与成本优化的AI应用开发者。
现有检索增强生成(RAG)系统多采用集中式架构,导致数据采集、整合与管理成本高,并引发隐私问题。亟需一种去中心化RAG系统,使大模型能直接从掌握数据所有权的用户提供信息。但去中心化带来挑战:各独立数据源可靠性差异大,影响检索准确性和响应质量。为此,我们提出一个去中心化RAG系统,引入新颖的可靠性评分机制,基于数据源贡献的回答质量动态评估其可信度,并在检索时优先使用高质量来源。评分过程通过区块链智能合约安全执行,实现可验证且防篡改的可靠性记录,无需依赖中心化机构。我们在两个模拟环境中,使用Llama 3B和8B模型测试,六个数据源具有不同可靠性水平。结果表明,在类似真实世界不可靠数据环境下,系统性能较中心化方案提升10.7%;在理想可靠数据下,逼近中心化系统的上限表现。去中心化基础设施还通过批量更新操作实现约56%的边际成本节省。代码与系统已开源至github.com/yining610/Reliable-dRAG。
原文摘要 · Abstract (English)
Existing retrieval-augmented generation (RAG) systems typically use a centralized architecture, causing a high cost of data collection, integration, and management, as well as privacy concerns. There is a great need for a decentralized RAG system that enables foundation models to utilize information directly from data owners who maintain full control over their sources. However, decentralization brings a challenge: the numerous independent data sources vary significantly in reliability, which can diminish retrieval accuracy and response quality. To address this, our decentralized RAG system has a novel reliability scoring mechanism that dynamically evaluates each source based on the quality of responses it contributes to generate and prioritizes high-quality sources during retrieval. To ensure transparency and trust, the scoring process is securely managed through blockchain-based smart contracts, creating verifiable and tamper-proof reliability records without relying on a central authority. We evaluate our decentralized system with two Llama models (3B and 8B) in two simulated environments where six data sources have different levels of reliability. Our system achieves a +10.7\% performance improvement over its centralized counterpart in the real world-like unreliable data environments. Notably, it approaches the upper-bound performance of centralized systems under ideally reliable data environments. The decentralized infrastructure enables secure and trustworthy scoring management, achieving approximately 56\% marginal cost savings through batched update operations. Our code and system are open-sourced at github.com/yining610/Reliable-dRAG.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。