动态数据下高效维护摘要式检索树，提升上下文质量
Recursive Abstractive Processing for Retrieval in Dynamic Datasets
- 构建递归摘要树并支持动态数据更新，无需重算
- 后置处理使检索上下文质量显著提升，性能更优
- 可作为通用模块适配任意检索算法，兼容性强
近期检索增强模型通过递归嵌入、聚类与摘要构建文本块的层次结构，从原始文本和生成摘要中检索最相关信息。然而，这类方法在动态数据集上面临挑战：文档增删导致聚类形成的层次结构难以更新。本文提出一种新算法，可在不损失性能的前提下高效维护递归摘要树结构。同时引入一种新的查询聚焦式后置递归摘要处理方法，显著提升上下文质量。所提方法作为黑箱后处理层，可兼容任意检索算法。在真实数据集上的大量实验验证了其在动态数据处理和检索性能提升方面的有效性。
原文摘要 · Abstract (English)
Recent retrieval-augmented models enhance basic methods by building a hierarchical structure over retrieved text chunks through recursive embedding, clustering, and summarization. The most relevant information is then retrieved from both the original text and generated summaries. However, such approaches face limitations with dynamic datasets, where adding or removing documents over time complicates the updating of hierarchical representations formed through clustering. We propose a new algorithm to efficiently maintain the recursive-abstractive tree structure in dynamic datasets, without compromising performance. Additionally, we introduce a novel post-retrieval method that applies query-focused recursive abstractive processing to substantially improve context quality. Our method overcomes the limitations of other approaches by functioning as a black-box post-retrieval layer compatible with any retrieval algorithm. Both algorithms are validated through extensive experiments on real-world datasets, demonstrating their effectiveness in handling dynamic data and improving retrieval performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。