用分段+压缩两阶段法，让大模型轻松总结超长对话。
A Novel LLM-based Two-stage Summarization Approach for Long Dialogues
- 先分段再压缩，把长文本拆成小块处理
- 用ChatGPT(v3.5)无监督生成压缩版内容
- 适合算力有限的场景，训练更快更省资源
长文档摘要在自然语言处理中面临重大挑战，因输入长度常超出主流预训练语言模型的容量。本文提出一种分层框架：先通过无监督话题分割方法识别语义合理断点，将长文档分段；再利用无监督生成模型（当前实验采用ChatGPT v3.5）生成压缩数据；最后在压缩文本上微调抽象摘要模型，生成最终摘要。该框架使模型即使在文档长度超过最大输入限制时仍可处理。由于摘要模型不直接接收完整文档，训练时间与计算资源消耗显著降低，适用于本地算力受限的场景。
原文摘要 · Abstract (English)
Long document summarization poses a significant challenge in natural language processing due to input lengths that exceed the capacity of most state-of-the-art pre-trained language models. This study proposes a hierarchical framework that segments and condenses information from long documents, subsequently fine-tuning the processed text with an abstractive summarization model. Unsupervised topic segmentation methods identify semantically appropriate breakpoints. The condensation stage utilizes an unsupervised generation model to generate condensed data, and our current experiments employ ChatGPT(v3.5). The summarization stage fine-tunes the abstractive summarization model on the condensed data to generate the final results. This framework enables long documents to be processed on models even when the document length exceeds the model's maximum input size. The exclusion of the entire document from the summarization model reduces the time and computational resources required for training, making the framework suitable for contexts with constrained local computational resources.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。