arXiv:2410.11001cs.CLcs.AI2024-10ACL被引 18

用历史生成结果构建图结构,提升长文本摘要的RAG效果

Graph of Records: Boosting Retrieval Augmented Generation for Long-context Summarization with Graphs

  • 将LLM的历史回复与检索文本连边,形成知识图谱增强信息关联
  • 在WCEP数据集上比基线提升15%~19%(Rouge-L/1/2)
  • 适合需要高精度长文档摘要的研究者和工业应用

检索增强生成(RAG)通过注入非参数化事实知识重振大语言模型(LLM)。相比长上下文LLM,RAG以更轻量、高效的方式,通过多轮查询交互获得全面响应。然而现有方法忽视了LLM生成的历史回复中蕴含的潜在信息,导致性能受限。本文提出图结构记录(GoR),利用历史生成响应增强长上下文全局摘要的RAG。受检索-生成范式启发,GoR将检索文本块与对应生成回复之间建立边,构建图结构。为挖掘其内在关联,引入图神经网络与基于BERTScore的自监督目标,实现参考摘要与节点嵌入间的梯度传递。在四个长文本摘要数据集上对比12个基线,结果表明GoR表现最佳(如在WCEP数据集上,Rouge-L、Rouge-1、Rouge-2分别提升15%、8%、19%)。大量实验验证了其有效性。

原文摘要 · Abstract (English)

Retrieval-augmented generation (RAG) has revitalized Large Language Models (LLMs) by injecting non-parametric factual knowledge. Compared with long-context LLMs, RAG is considered an effective summarization tool in a more concise and lightweight manner, which can interact with LLMs multiple times using diverse queries to get comprehensive responses. However, the LLM-generated historical responses, which contain potentially insightful information, are largely neglected and discarded by existing approaches, leading to suboptimal results. In this paper, we propose $\textit{graph of records}$ ($\textbf{GoR}$), which leverages historical responses generated by LLMs to enhance RAG for long-context global summarization. Inspired by the $\textit{retrieve-then-generate}$ paradigm of RAG, we construct a graph by establishing an edge between the retrieved text chunks and the corresponding LLM-generated response. To further uncover the intricate correlations between them, GoR features a $\textit{graph neural network}$ and an elaborately designed $\textit{BERTScore}$-based objective for self-supervised model training, enabling seamless supervision signal backpropagation between reference summaries and node embeddings. We comprehensively compare GoR with 12 baselines across four long-context summarization datasets, and the results indicate that our proposed method reaches the best performance ($\textit{e.g.}$, 15%, 8%, and 19% improvement over retrievers w.r.t. Rouge-L, Rouge-1, and Rouge-2 on the WCEP dataset). Extensive experiments further demonstrate the effectiveness of GoR.

RAG长文本摘要图神经网络

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。