让大模型自动调整摘要用的关键词,更贴合文档内容。
CARPAS: Towards Content-Aware Refinement of Provided Aspects for Summarization in Large Language Models
- 根据文档内容动态优化给定的摘要关键词
- 预测相关关键词数量可显著提升摘要质量
- 适合需要自适应摘要的智能系统开发者
基于关键词的摘要技术因能生成更细致、更符合用户需求的摘要而受到关注。现有方法通常假设输入关键词是预设好的,但在实际场景中,这些关键词可能不完整、无关或缺失。用户期望系统能根据内容自动调整或筛选关键词。本文首次提出内容感知的关键词优化任务(CARPAS),旨在总结前动态调整输入关键词。我们构建了三个新数据集用于初步实验,发现大模型在提示下常生成过多关键词,导致摘要过长且偏离重点。基于此,我们提出预测有效关键词数量的子任务,结果表明该数量可作为有效引导,降低推理难度,帮助模型聚焦核心信息。大量实验证明该方法在所有数据集上均有显著提升。深入分析还发现,当要求关键词数与模型自发估计不一致时,模型仍具可遵循性,为实际部署提供关键洞见。
原文摘要 · Abstract (English)
Aspect-based summarization has attracted significant attention for its ability to generate more fine-grained and user-aligned summaries. While most existing approaches assume a set of predefined aspects as input, real-world scenarios often present challenges where these given aspects may be incomplete, irrelevant, or entirely missing from the document. Users frequently expect systems to adaptively refine or filter the provided aspects based on the actual content. In this paper, we initiate this novel task setting, termed Content-Aware Refinement of Provided Aspects for Summarization (CARPAS), with the aim of dynamically adjusting the provided aspects based on the document context before summarizing. We construct three new datasets to facilitate our pilot experiments, and by using LLMs with four representative prompting strategies in this task, we find that LLMs tend to predict an overly comprehensive set of aspects, which often results in excessively long and misaligned summaries. Building on this observation, we propose a preliminary subtask to predict the number of relevant aspects, and demonstrate that the predicted number can serve as effective guidance for the LLMs, reducing the inference difficulty, and enabling them to focus on the most pertinent aspects. Our extensive experiments show that the proposed approach significantly improves performance across all datasets. Moreover, our deeper analyses uncover LLMs' compliance when the requested number of aspects differs from their own estimations, establishing a crucial insight for the deployment of LLMs in similar real-world applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。