针对长文档摘要难题,提出分页对齐与重要性加权新方法。
Long document summarization using page specific target text alignment and distilling page importance
- 将长文档分页,每页与摘要相关部分对齐以增强监督
- 引入动态页重要性权重,使模型聚焦关键内容
- 在基准数据集上超越当前最优结果6.32%(ROUGE-1)
新闻、法律、医疗和科学领域文本数据的快速增长,使得高效获取和理解大量内容成为挑战。用户难以快速提取有效信息,因此亟需自动摘要技术。与短文档摘要不同,长文档抽象式摘要资源消耗大,研究较少。虽然BART是广泛使用的序列到序列模型,但其上下文窗口长度限制了处理长文档的能力。我们提出PTS(Page-specific Target-text alignment Summarization)模型,通过将源文档分页,使每页与目标摘要的相关部分对齐,实现更好的监督,并生成各页的局部摘要。进一步提出PTSPI模型,在合并局部摘要前增加一个层,提供动态页重要性权重和显式监督,强化对关键页的关注。在基准数据集上的实验表明,PTSPI在ROUGE-1上优于当前最优模型6.32%,在ROUGE-2上提升8.08%。
原文摘要 · Abstract (English)
The rapid growth of textual data across news, legal, medical, and scientific domains is becoming a challenge for efficiently accessing and understanding large volumes of content. It is increasingly complex for users to consume and extract meaningful information efficiently. Thus, raising the need for summarization. Unlike short document summarization, long document abstractive summarization is resource-intensive, and very little literature is present in this direction. BART is a widely used efficient sequence-to-sequence (seq-to-seq) model. However, when it comes to summarizing long documents, the length of the context window limits its capabilities. We proposed a model called PTS (Page-specific Target-text alignment Summarization) that extends the seq-to-seq method for abstractive summarization by dividing the source document into several pages. PTS aligns each page with the relevant part of the target summary for better supervision. Partial summaries are generated for each page of the document. We proposed another model called PTSPI (Page-specific Target-text alignment Summarization with Page Importance), an extension to PTS where an additional layer is placed before merging the partial summaries into the final summary. This layer provides dynamic page weightage and explicit supervision to focus on the most informative pages. We performed experiments on the benchmark dataset and found that PTSPI outperformed the SOTA by 6.32\% in ROUGE-1 and 8.08\% in ROUGE-2 scores.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。