用图模型和信念传播实现无监督长文本分割，兼顾相邻句与远距离语义相似句的分组。
BP-Seg: A graphical model approach to unsupervised and non-contiguous text segmentation using belief propagation
- 构建图模型，通过信念传播融合局部连贯性与远距离语义关联。
- 在长文档数据集上优于现有方法，能有效识别非连续语义段落。
- 适合需要无监督语义分段的文本分析任务，如摘要生成、信息抽取。
基于句子语义的文本分割是众多下游应用的基础任务。本文提出一种基于图模型的无监督学习方法BP-Seg，实现高效文本分割。该方法不仅考虑局部连贯性（相邻句子更相关），还能有效将文本中相距较远但语义相似的句子分组。这一能力通过在精心构建的图模型上进行信念传播实现。在示例案例和包含长篇文档的数据集上的实验结果表明，该方法相比现有方法表现更优。
原文摘要 · Abstract (English)
Text segmentation based on the semantic meaning of sentences is a fundamental task with broad utility in many downstream applications. In this paper, we propose a graphical model-based unsupervised learning approach, named BP-Seg for efficient text segmentation. Our method not only considers local coherence, capturing the intuition that adjacent sentences are often more related, but also effectively groups sentences that are distant in the text yet semantically similar. This is achieved through belief propagation on the carefully constructed graphical models. Experimental results on both an illustrative example and a dataset with long-form documents demonstrate that our method performs favorably compared to competing approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。