解决多文档检索中相似内容混淆问题，提升生成准确性。
CHOP: Chunkwise Context-Preserving Framework for RAG on Multi Documents
- 用大模型逐块评估相关性，动态重构文档结构。
- 在基准数据集上实现90.77%的Top-1命中率，排名指标显著提升。
- 适合构建高质量知识库，尤其适用于复杂多文档场景。
当向量数据库中存在相似文档时，检索增强生成（RAG）系统会降低检索准确率，导致冗余信息、幻觉和事实错误。为此，我们提出CHOP框架，通过大语言模型（LLM）迭代评估分块相关性，并逐步重建文档以确定其与特定主题或查询类型的关系。CHOP包含两个核心组件：CNM-Extractor生成包含类别、关键词和模型名的紧凑分块签名；连续性决策模块通过判断相邻分块是否属于同一文档流来保持上下文连贯性。通过在每段前添加上下文感知元数据，CHOP减少了相似文档间的语义冲突，提升了检索器区分能力。在基准数据集上的实验表明，CHOP有效缓解了检索混淆问题，为构建高质量知识库提供了可扩展方案，实现了90.77%的Top-1命中率，并在排序质量指标上取得显著提升。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) systems lose retrieval accuracy when similar documents coexist in the vector database, causing unnecessary information, hallucinations, and factual errors. To alleviate this issue, we propose CHOP, a framework that iteratively evaluates chunk relevance with Large Language Models (LLMs) and progressively reconstructs documents by determining their association with specific topics or query types. CHOP integrates two key components: the CNM-Extractor, which generates compact per-chunk signatures capturing categories, key nouns, and model names, and the Continuity Decision Module, which preserves contextual coherence by deciding whether consecutive chunks belong to the same document flow. By prefixing each chunk with context-aware metadata, CHOP reduces semantic conflicts among similar documents and enhances retriever discrimination. Experiments on benchmark datasets show that CHOP alleviates retrieval confusion and provides a scalable approach for building high-quality knowledge bases, achieving a Top-1 Hit Rate of 90.77% and notable gains in ranking quality metrics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。