利用大模型自注意力机制自动提取关键词，无需调参。
Attention-Seeker: Dynamic Self-Attention Scoring for Unsupervised Keyphrase Extraction
- 从大模型自注意力图中动态筛选关键层、头和向量
- 在四个数据集上均超越多数基线模型，长文档效果尤佳
- 完全无需人工调参，适合实际应用
本文提出 Attention-Seeker，一种无监督关键词提取方法，利用大语言模型的自注意力图来评估候选短语的重要性。该方法识别模型在文本关键主题上关注度高的特定组件——如层、注意力头和注意力向量，并使用这些组件提供的注意力权重对候选短语进行评分。与以往需手动调整参数（如选择注意力头、提示词、超参数）的方法不同，Attention-Seeker 能够根据输入文本动态适应，无需任何人工调整，显著提升实用性。我们在 Inspec、SemEval2010、SemEval2017 和 Krapivin 四个公开数据集上进行了评估。结果表明，即使不进行参数调优，Attention-Seeker 在三个数据集上达到当前最优性能，尤其在长文档关键词提取任务中表现突出。
原文摘要 · Abstract (English)
This paper proposes Attention-Seeker, an unsupervised keyphrase extraction method that leverages self-attention maps from a Large Language Model to estimate the importance of candidate phrases. Our approach identifies specific components - such as layers, heads, and attention vectors - where the model pays significant attention to the key topics of the text. The attention weights provided by these components are then used to score the candidate phrases. Unlike previous models that require manual tuning of parameters (e.g., selection of heads, prompts, hyperparameters), Attention-Seeker dynamically adapts to the input text without any manual adjustments, enhancing its practical applicability. We evaluate Attention-Seeker on four publicly available datasets: Inspec, SemEval2010, SemEval2017, and Krapivin. Our results demonstrate that, even without parameter tuning, Attention-Seeker outperforms most baseline models, achieving state-of-the-art performance on three out of four datasets, particularly excelling in extracting keyphrases from long documents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。