让命名实体识别给出带置信度的预测集,防错更可靠
Uncertainty Quantification for Named Entity Recognition via Full-Sequence and Subsequence Conformal Prediction
- 用全序列与子序列校准法生成带置信度的标签集合
- 在3个数据集上验证,预测集覆盖率达用户指定水平
- 适合对可靠性要求高的医疗、金融等场景应用
命名实体识别(NER)是自然语言处理中的基础任务,但现有模型仅输出单一标签序列,缺乏不确定性度量,易导致下游应用出现级联错误。本文提出一种通用框架,将序列标注型NER模型改造为输出不确定性感知的预测集。这些预测集是包含正确标注的完整句子标签组合,且在用户设定的置信水平下具有形式化保证,类似经典统计中的置信区间。方法基于合取预测(conformal prediction),在最小假设下提供有限样本覆盖率。设计高效的非一致性评分函数,构建出具有良好校准性的预测集,支持无条件和类别条件覆盖。该框架能适应句长、语言、实体类型及句内实体数量的异质性。在四个NER模型和三个基准数据集上的实验表明,该方法具备广泛适用性、有效性与高效性。
原文摘要 · Abstract (English)
Named Entity Recognition (NER) serves as a foundational component in many natural language processing (NLP) pipelines. However, current NER models typically output a single predicted label sequence without any accompanying measure of uncertainty, leaving downstream applications vulnerable to cascading errors. In this paper, we introduce a general framework for adapting sequence-labeling-based NER models to produce uncertainty-aware prediction sets. These prediction sets are collections of full-sentence labelings that are guaranteed to contain the correct labeling with a user-specified confidence level. This approach serves a role analogous to confidence intervals in classical statistics by providing formal guarantees about the reliability of model predictions. Our method builds on conformal prediction, which offers finite-sample coverage guarantees under minimal assumptions. We design efficient nonconformity scoring functions to construct efficient, well-calibrated prediction sets that support both unconditional and class-conditional coverage. This framework accounts for heterogeneity across sentence length, language, entity type, and number of entities within a sentence. Empirical experiments on four NER models across three benchmark datasets demonstrate the broad applicability, validity, and efficiency of the proposed methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。