arXiv:2409.11827cs.CL2024-09被引 3

用单一模型统一抽取与摘要,提升生成质量并减少误差积累。

Extract-and-Abstract: Unifying Extractive and Abstractive Summarization within Single Encoder-Decoder Framework

  • 用显著性掩码替代注意力掩码,让解码器专注关键信息。
  • 在三个数据集上,抽取任务性能超越基线,摘要任务持平或更优。
  • 无需额外参数,可无缝融合抽取与摘要,适合追求高效生成的场景。

提取-摘要是一种自然连贯的摘要生成范式,借助抽取模型识别的关键信息来辅助抽象摘要。以往方法分别训练抽取器与摘要器,并引入额外参数突出抽取内容,导致误差累积和额外训练成本。本文首次在编码器-解码器框架中引入无参数的显著性提示方法:将交叉注意力模块中的编码器注意力掩码替换为显著性掩码,强制解码器仅关注输入中的关键部分。初步分析对比了多种提示方法,验证了显著性掩码的有效性。进一步提出新型提取-摘要范式ExtAbs,将抽取与摘要任务在单个编码器-解码器模型中联合、无缝完成,降低误差传播。在ExtAbs中,原始编码器被增强以抽取关键内容,原始解码器通过提出的显著性掩码进行摘要生成。基于BART与PEGASUS,在三个数据集上的实验表明,ExtAbs在抽取任务上优于基线,在摘要任务上表现相当甚至更优。

原文摘要 · Abstract (English)

Extract-then-Abstract is a naturally coherent paradigm to conduct abstractive summarization with the help of salient information identified by the extractive model. Previous works that adopt this paradigm train the extractor and abstractor separately and introduce extra parameters to highlight the extracted salients to the abstractor, which results in error accumulation and additional training costs. In this paper, we first introduce a parameter-free highlight method into the encoder-decoder framework: replacing the encoder attention mask with a saliency mask in the cross-attention module to force the decoder to focus only on salient parts of the input. A preliminary analysis compares different highlight methods, demonstrating the effectiveness of our saliency mask. We further propose the novel extract-and-abstract paradigm, ExtAbs., which jointly and seamlessly performs Extractive and Abstractive summarization tasks within single encoder-decoder model to reduce error accumulation. In ExtAbs, the vanilla encoder is augmented to extract salients, and the vanilla decoder is modified with the proposed saliency mask to generate summaries. Built upon BART and PEGASUS, experiments on three datasets show that ExtAbs can achieve superior performance than baselines on the extractive task and performs comparable, or even better than the vanilla models on the abstractive task.

摘要生成统一框架显著性掩码BART

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。