通过自监督学习挖掘句子中的事实结构,提升知识型NLP任务性能
Sentence Splitter: Uncovering Latent Factual Structure for Self-Supervised Learning

- 将句子分割建模为离散分段问题,用概率生成恢复事实补全
- 在知识图谱补全和常识问答任务上显著提升性能
- 无需人工标注,可自动构建结构化训练数据,适合知识增强场景
本文提出Sentence Splitter,一种基于T5的自监督框架,用于揭示自然语言句子中的潜在事实结构。该方法将句子分割视为离散分段问题:长度为N的句子有N个可能的分割点,但仅一个能正确划分描述前缀(head)与事实补全(tail)。模型不显式搜索所有候选边界,而是通过概率序列生成恢复事实补全。为避免人工标注,先将符号化的头尾对转化为自然语言模板作为训练监督信号。训练后的分割器应用于原始文本,提取对齐的前缀-尾部对,并用于轻量级自举过程,训练生成模型以提出更多合理补全。该统一流程实现了可扩展的结构感知训练数据构建,有效连接符号知识与自然语言。在结构化与自然文本上的实验表明,该分割器超越合成模板泛化能力,且生成的结构感知监督持续提升知识图谱补全与常识问答的下游性能,验证了挖掘潜在事实结构在知识导向NLP中的有效性。
原文摘要 · Abstract (English)
This paper introduces Sentence Splitter, a self-supervised framework built upon a T5-based encoder--decoder architecture for uncovering the latent factual structure of natural language sentences. The proposed method identifies the semantic boundary between a descriptive prefix (head) and its factual completion (tail) by formulating sentence splitting as a discrete segmentation problem, where a sentence of length $N$ admits $N$ possible split points but only one recovers the intended head--tail structure. Rather than explicitly searching over all candidate boundaries, the model learns to recover the factual completion through probabilistic sequence generation. To eliminate the need for manual annotation, symbolic head--tail pairs are first verbalized into natural-language templates that provide supervision for training the Sentence Splitter. The trained splitter is then applied to raw text to extract aligned prefix--tail pairs, which are subsequently used to train a generative model that proposes additional plausible completions through a lightweight bootstrapping process. This unified pipeline provides a scalable and structure-aware approach to constructing self-supervised training data while bridging symbolic knowledge and natural language. Experiments on both structured and naturally occurring text demonstrate that the proposed splitter generalizes beyond synthetic templates and that the resulting structure-aware supervision consistently improves downstream performance on knowledge graph completion and commonsense question answering, highlighting the effectiveness of recovering latent factual structure for knowledge-centric NLP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。