提出新方法与数据集，实现文本实体重要性分级评分。
GUM-SAGE: A Novel Dataset and Approach for Graded Entity Salience Prediction
- 基于多份摘要中实体出现频率计算重要性得分
- 在12类语料上相关性优于现有方法，包括大模型
- 适合需要细粒度实体排序的阅读理解系统
确定文本中最重要的实体并进行排序对面向用户的系统至关重要，尤其当用户依赖模型解读他们只读了一部分的长文档时。分级实体显著性通过分配反映实体相对重要性的分数来满足这一需求。现有方法分为两类：主观判断型，虽可梯度评分但一致性差；基于摘要的方法，将显著性定义为摘要中的提及价值，提升可解释性但仅限二元标签。本文提出一种结合两者优势的新方法，使用涵盖12种口语和书面体裁的英文数据集，每篇文档生成5份摘要，根据实体在摘要中的出现情况计算其显著性得分。该方法在人类摘要与对齐评分上的相关性更强，性能超越现有技术，包括大语言模型。我们已在https://github.com/jl908069/gum_sum_salience发布数据与代码，以支持分级显著实体提取的进一步研究。
原文摘要 · Abstract (English)
Determining and ranking the most salient entities in a text is critical for user-facing systems, especially as users increasingly rely on models to interpret long documents they only partially read. Graded entity salience addresses this need by assigning entities scores that reflect their relative importance in a text. Existing approaches fall into two main categories: subjective judgments of salience, which allow for gradient scoring but lack consistency, and summarization-based methods, which define salience as mention-worthiness in a summary, promoting explainability but limiting outputs to binary labels (entities are either summary-worthy or not). In this paper, we introduce a novel approach for graded entity salience that combines the strengths of both approaches. Using an English dataset spanning 12 spoken and written genres, we collect 5 summaries per document and calculate each entity's salience score based on its presence across these summaries. Our approach shows stronger correlation with scores based on human summaries and alignments, and outperforms existing techniques, including LLMs. We release our data and code at https://github.com/jl908069/gum_sum_salience to support further research on graded salient entity extraction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。