用关键短语增强提示，让大模型生成更完整准确的摘要。
Salient Information Prompting to Steer Content in Prompt-based Abstractive Summarization
- 从原文提取关键短语加入提示，引导模型生成
- 提升ROUGE F1和召回率，关键短语数量可调精度与召回平衡
- 轻量级模型SigExt可适配各类大模型，效果稳定
大语言模型可通过提示技术生成跨领域的流畅摘要，减少专门训练的需求。但设计能有效引导模型生成适当细节和写作风格的提示仍具挑战。本文探索利用源文档中的显著信息（关键短语）增强提示。实验表明，在提示中加入关键短语可提升ROUGE F1和召回率，使生成摘要更接近参考文本且更完整；关键短语数量可控制精确率与召回率的权衡。分析还发现，短语级显著信息优于词级或句级。然而，对幻觉的影响在不同大模型间不一致。为此，我们提出轻量级关键短语提取器SigExt，可微调以提取显著短语。使用SigExt可在多个数据集及开源与专有大模型上实现一致的ROUGE提升，无需定制大模型。研究为构建基于提示的摘要系统提供了新见解。代码已开源。
原文摘要 · Abstract (English)
Large language models (LLMs) can generate fluent summaries across domains using prompting techniques, reducing the need to train models for summarization applications. However, crafting effective prompts that guide LLMs to generate summaries with the appropriate level of detail and writing style remains a challenge. In this paper, we explore the use of salient information extracted from the source document to enhance summarization prompts. We show that adding keyphrases in prompts can improve ROUGE F1 and recall, making the generated summaries more similar to the reference and more complete. The number of keyphrases can control the precision-recall trade-off. Furthermore, our analysis reveals that incorporating phrase-level salient information is superior to word- or sentence-level. However, the impact on hallucination is not universally positive across LLMs. To conduct this analysis, we introduce Keyphrase Signal Extractor (SigExt), a lightweight model that can be finetuned to extract salient keyphrases. By using SigExt, we achieve consistent ROUGE improvements across datasets and open-weight and proprietary LLMs without any LLM customization. Our findings provide insights into leveraging salient information in building prompt-based summarization systems. We release our code at \url{https://github.com/amazon-science/SigExt}
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。