让多个AI代理协同优化，提升问答系统准确率。
GRADRAG: Cross-Component Prompt Adaptation for Coordinated Multi-Agent RAG
- 将RAG流程建模为计算图，通过反馈反向调整各组件提示词。
- 在两个基准上比单步优化方法高出12-15个百分点的偏好得分。
- 适合需要多智能体协作的复杂问答系统研发者使用。
检索增强生成（RAG）系统越来越多地采用多个大语言模型代理。然而，以往工作通常孤立优化各组件，而非协调整个流程的改进。我们提出GRADRAG框架，实现跨组件提示词自适应，将RAG管道建模为计算图，并将结构化评估反馈传播至上游代理。评估器对下游答案和支撑证据进行评判，生成可操作的反馈，由提示词优化器用于迭代更新检索器、图构建器和回答器等自适应代理。评估器还可在输出达标时触发早停。我们在SQUALITY和QMSUM基准上，分别在基于扁平块的检索（使用IRCoT风格查询优化）和基于图的检索（从文档构建并迭代丰富实体-关系图）两种范式下评估GRADRAG。在两种设置中，GRADRAG均持续优于仅更新最终生成器的一次性优化基线，在大模型判断的成对比较中实现12-15个百分点的净偏好优势，且大部分增益在两次优化迭代内达成。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) systems increasingly employ multiple LLM agents. Yet, most prior work optimizes components in isolation rather than coordinating improvements across the pipeline. We introduce GRADRAG, a framework for cross-component prompt adaptation that models the RAG pipeline as a computational graph and propagates structured evaluation feedback to update upstream agents. An Evaluator critiques downstream answers and supporting evidence, producing actionable feedback that a Prompt Optimizer uses to iteratively update adaptive agents, such as retrievers, graph constructors, and answerers. The Evaluator also triggers early stopping when the output is deemed satisfactory. We evaluate GRADRAG on the SQUALITY and QMSUM benchmarks under two retrieval paradigms: flat chunk-based retrieval using IRCoT-style query refinement (Trivedi et al., 2023), and graph-based retrieval that constructs and iteratively enriches an entity-relation graph from the document. Across both settings, GRADRAG consistently outperforms one-step refinement baselines that update only the final generator, achieving a 12-15 percentage point net preference margin in LLM-judged pairwise comparisons, with most gains realized within two refinement iterations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。