用跨段语义融合提升文档分段准确率
CrossFormer: Cross-Segment Semantic Fusion for Document Segmentation
- 引入动态跨段融合模块，捕捉段落间潜在语义关联
- 在多个公开数据集上达到当前最优性能，显著提升RAG效果
- 适合需要精准文本分割的智能问答与信息抽取场景
文本语义分段旨在根据主题、上下文和文档结构将文档划分为语义连贯的段落。传统方法通常将文档预分割以应对输入长度限制，导致跨段关键语义信息丢失。为此，我们提出CrossFormer，一种基于Transformer的模型，其包含新颖的跨段融合模块，可动态建模文档段落间的潜在语义依赖关系，显著提升分段准确率。此外，CrossFormer可替代检索增强生成（RAG）系统中的规则分块方法，生成更语义连贯的文本块，从而增强系统效能。全面评估表明，CrossFormer在多个公开文本语义分段数据集上表现领先，并在RAG基准测试中取得显著提升。
原文摘要 · Abstract (English)
Text semantic segmentation involves partitioning a document into multiple paragraphs with continuous semantics based on the subject matter, contextual information, and document structure. Traditional approaches have typically relied on preprocessing documents into segments to address input length constraints, resulting in the loss of critical semantic information across segments. To address this, we present CrossFormer, a transformer-based model featuring a novel cross-segment fusion module that dynamically models latent semantic dependencies across document segments, substantially elevating segmentation accuracy. Additionally, CrossFormer can replace rule-based chunk methods within the Retrieval-Augmented Generation (RAG) system, producing more semantically coherent chunks that enhance its efficacy. Comprehensive evaluations confirm CrossFormer's state-of-the-art performance on public text semantic segmentation datasets, alongside considerable gains on RAG benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。