通过迭代提问生成更易被大模型理解的浓缩摘要
Chain of Summaries: Summarization Through Iterative Questioning
- 用提问发现摘要缺陷，逐步优化成通用摘要
- 在多个数据集上比基线提升最高66%
- 摘要更短且适配任意下游模型，适合网站内容改造
大型语言模型（LLMs）越来越多地依赖外部网络内容，但这些内容常因格式不友好和上下文长度限制难以被模型有效利用。为此，我们提出链式摘要（Chain of Summaries, CoS）方法，通过迭代方式生成通用、信息密集的纯文本摘要，作为网页内容的可读仓库。受黑格尔辩证法启发，CoS 从初始摘要（正题）出发，通过提问识别其局限性（反题），最终形成能应对当前与未来需求的通用摘要（合题）。在 TriviaQA、TruthfulQA 和 SQUAD 数据集上的实验表明，CoS 比零样本 LLM 基线最高提升 66%，比 Chain of Density、BRIO、PEGASUS 等专门摘要方法最高提升 27%。生成的摘要在问答任务中表现优于原始内容，且所需标记数显著减少，对特定下游模型无依赖性。因此，CoS 为网站维护者提供了一种使内容更易被大模型访问的可行方案，同时保留人工审核空间。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are increasingly using external web content. However, much of this content is not easily digestible by LLMs due to LLM-unfriendly formats and limitations of context length. To address this issue, we propose a method for generating general-purpose, information-dense summaries that act as plain-text repositories of web content. Inspired by Hegel's dialectical method, our approach, denoted as Chain of Summaries (CoS), iteratively refines an initial summary (thesis) by identifying its limitations through questioning (antithesis), leading to a general-purpose summary (synthesis) that can satisfy current and anticipate future information needs. Experiments on the TriviaQA, TruthfulQA, and SQUAD datasets demonstrate that CoS outperforms zero-shot LLM baselines by up to 66\% and specialized summarization methods such as Chain of Density, BRIO and PEGASUS by up to 27\%. CoS-generated summaries yield higher Q\&A performance compared to the source content, while requiring substantially fewer tokens and being agnostic to the specific downstream LLM. CoS thus resembles an appealing option for website maintainers to make their content more accessible for LLMs, while retaining possibilities for human oversight.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。