基于叙事理论的文学文档分割,提升长尾文学问答效果
LitSeg: Narrative-Aware Document Segmentation for Literary RAG

- 用多阶段提示提取事件、理清叙事线、定位转折点指导分段
- 相比基线,检索准确率与上下文相关性显著提升,问答性能增强
- 轻量版可单次推理,适合大规模文学文本处理
检索增强生成(RAG)通过引入外部知识增强大语言模型能力,尤其适用于长尾领域如文学作品。然而,RAG中的文档分割步骤仍缺乏深入研究。现有方法通常忽略文学作品复杂的叙事结构,导致情节碎片化和指代不清,严重影响检索与生成效果。为此,我们提出LitSeg,一种基于叙事理论的分段框架。通过多阶段提示,显式提取有效事件、梳理叙事脉络、明确结构并定位转折点以指导分段。为缓解大规模模型多阶段推理带来的计算开销,我们进一步提出LitSeg-Lite,一个在LitSeg生成数据上通过两阶段训练微调的轻量级单次推理分块器,将复杂流程压缩为一次推断。大量实验表明,使用结构独立的文本块,我们的方法显著提升检索准确率与上下文相关性,最终改善下游问答性能;消融实验证明了叙事引导与数据蒸馏的有效性。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) enhances Large Language Models (LLMs) by incorporating external knowledge, particularly for long-tail domains such as literary works. However, the critical step of document segmentation in RAG remains largely underexplored. Existing strategies are typically semantically blind and overlook the complicated narrative structures of literary works, often resulting in fragmented plots and unclear references that severely hinder retrieval and generation performance. To address this, we propose LitSeg, a novel narrative-theory-guided segmentation framework. By employing multi-stage prompting, LitSeg explicitly extracts valid events, untangles narrative threads, clarifies narrative structures, and locates turning points to inform segmentation. To alleviate the computational overhead of multi-stage inference with large-scale models, we further introduce LitSeg-Lite, a lightweight single-pass chunker fine-tuned on LitSeg-generated data via a two-stage training strategy, distilling the complex process into a single inference pass. Extensive experiments demonstrate that with structurally independent text chunks, our methods significantly improve retrieval accuracy and context relevance over baselines, ultimately enhancing downstream QA performance, while ablation studies validate the efficacy of narratological guidance and data distillation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。