提出新评估框架与分层切块方法，提升RAG系统知识检索效果
HiChunk: Evaluating and Enhancing Retrieval-Augmented Generation with Hierarchical Chunking
- 构建多层级文档切块评估基准HiCBench，解决证据稀疏问题
- 提出基于微调大模型的分层切块框架，提升检索准确率23.5%以上
- 适合关注RAG知识检索优化的研究者与工程师
检索增强生成（RAG）通过整合外部知识源提升语言模型的响应能力。然而，作为RAG系统重要环节的文档切块缺乏有效的评估工具。本文分析现有RAG评估基准在评估文档切块质量时因证据稀疏而不足的问题，提出HiCBench基准，包含人工标注的多层级文档切块点、合成的高证据密度问答对及其对应证据来源。此外，提出基于微调大模型的分层切块框架HiChunk，结合Auto-Merge检索算法以提升检索质量。实验表明，HiCBench能有效评估不同切块方法在整个RAG流程中的影响；同时，HiChunk在合理时间开销下实现更优的切块质量，显著提升RAG系统整体性能。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) enhances the response capabilities of language models by integrating external knowledge sources. However, document chunking as an important part of RAG system often lacks effective evaluation tools. This paper first analyzes why existing RAG evaluation benchmarks are inadequate for assessing document chunking quality, specifically due to evidence sparsity. Based on this conclusion, we propose HiCBench, which includes manually annotated multi-level document chunking points, synthesized evidence-dense quetion answer(QA) pairs, and their corresponding evidence sources. Additionally, we introduce the HiChunk framework, a multi-level document structuring framework based on fine-tuned LLMs, combined with the Auto-Merge retrieval algorithm to improve retrieval quality. Experiments demonstrate that HiCBench effectively evaluates the impact of different chunking methods across the entire RAG pipeline. Moreover, HiChunk achieves better chunking quality within reasonable time consumption, thereby enhancing the overall performance of RAG systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。