通过分步筛选关键点提升多文档摘要的覆盖多样性。
Principled Content Selection to Generate Diverse and Personalized Multi-Document Summaries
- 分三步：提取关键点→用DPP选多样化内容→重写成摘要
- 在DiverseSumm上显著提升不同LLM的源文档覆盖度
- 可结合用户意图生成个性化摘要，兼顾相关性与覆盖面
尽管大语言模型（LLMs）能处理更长上下文，但近期研究发现其存在‘中间遗忘’现象（Liu et al., 2024），对输入内容各部分关注度不均。这限制了多文档摘要中对多样化来源材料的覆盖能力，如DiverseSumm基准测试所揭示。本文认为，有原则的内容选择是提升该任务覆盖率的简单有效方法。不同于直接一步式提示，我们明确将任务分为三步：(1) 将文档集合简化为原子级关键点；(2) 使用确定性点过程（DPP）选择优先多样性的关键点；(3) 重写生成最终摘要。通过结合提示工程与有原则的选择技术，我们在多种LLM上一致提升了DiverseSumm基准上的源覆盖度。此外，通过将用户意图融入DPP核函数，我们还能生成既覆盖相关源信息又保持多样性的个性化摘要。
原文摘要 · Abstract (English)
While large language models (LLMs) are increasingly capable of handling longer contexts, recent work has demonstrated that they exhibit the "lost in the middle" phenomenon (Liu et al., 2024) of unevenly attending to different parts of the provided context. This hinders their ability to cover diverse source material in multi-document summarization, as noted in the DiverseSumm benchmark (Huang et al., 2024). In this work, we contend that principled content selection is a simple way to increase source coverage on this task. As opposed to prompting an LLM to perform the summarization in a single step, we explicitly divide the task into three steps -- (1) reducing document collections to atomic key points, (2) using determinantal point processes (DPP) to perform select key points that prioritize diverse content, and (3) rewriting to the final summary. By combining prompting steps, for extraction and rewriting, with principled techniques, for content selection, we consistently improve source coverage on the DiverseSumm benchmark across various LLMs. Finally, we also show that by incorporating relevance to a provided user intent into the DPP kernel, we can generate personalized summaries that cover relevant source information while retaining coverage.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。