arXiv:2507.05235cs.LGcs.CL2025-07被引 1

通过重加权关键词概率,让生成摘要更聚焦主题。

Logit Reweighting for Topic-Focused Summarization

  • 生成时直接调整关键词的逻辑值(logits)以增强主题相关性
  • 阈值选择法提升主题专注度,且不降低摘要质量
  • 适合资源有限的小模型实现精准主题控制

生成符合特定主题的抽象摘要仍是语言模型的重大挑战。标准方法如微调成本高,而提示工程等简单方法在小模型上难以保持主题聚焦。为此,我们提出一种轻量级方法:在生成过程中直接重加权与主题相关的词元逻辑值。评估了三种重加权技术:常数偏移(Constant Shift)、因子缩放(Factor Scaling)和阈值选择(Threshold Selection)。在 NEWTS 主题摘要数据集上,使用 Gemma-2B 和 Llama-3-8B 模型进行实验,结果表明这些方法能有效提升主题相关词汇的使用率。尤其值得注意的是,阈值选择法在不损害摘要质量的前提下显著增强主题聚焦,克服了以往方法中主题与质量之间的权衡。研究证明,直接重加权逻辑值是一种高效、低成本的替代微调方案,为精确控制生成文本的主题内容提供了可行路径。

原文摘要 · Abstract (English)

Generating abstractive summaries that adhere to a specific topic remains a significant challenge for language models. While standard approaches, such as fine-tuning, are resource-intensive, simpler methods like prompt engineering often struggle to maintain topical focus, particularly with smaller models. To address this, we propose a lightweight method that enhances topical relevance by directly reweighting the logits of topic-relevant tokens during generation. We evaluate three such reweighting techniques: Constant Shift, which adds a constant value to logits; Factor Scaling, which multiplies them by a factor; and Threshold Selection, which selectively boosts logits that exceed a probability threshold. Experiments on the NEWTS topical summarization dataset, using both Gemma-2B and Llama-3-8B models, show that these techniques effectively increase the use of topic-relevant vocabulary. Notably, the Threshold Selection method successfully improves topical focus without compromising summary quality-a trade-off often seen in other approaches. Our findings demonstrate that directly reweighting logits is a practical and resource-efficient alternative to fine-tuning, offering a promising pathway for precisely controlling the thematic content of generated text.

摘要生成主题聚焦轻量化方法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。