基于训练动态自动清洗噪声标签,显著提升远程监督实体识别效果
DynClean: Training Dynamics-based Label Cleaning for Distantly-Supervised Named Entity Recognition
- 通过模型训练过程中的行为差异识别错误标签
- 清洗后模型F1提升3.18%至8.95%,在四个数据集上优于主流方法
- 无需人工干预,适合大规模远程监督场景
远程监督命名实体识别(DS-NER)因其可扩展性和自动生成标注数据的能力受到关注。但远程标注引入大量错误标签,限制了其性能。现有工作多通过复杂模型学习噪声标签,而对数据清洗的关注较少。本文提出一种基于训练动态的标签清洗方法,利用模型训练过程中的行为特征刻画远距离标注样本,并设计自动阈值估计策略定位错误标签。大量实验表明:(1)在经清洗后的DS-NER数据集上训练的模型,F1得分提升3.18%至8.95%;(2)本方法在四个数据集上优于多种先进DS-NER方法。
原文摘要 · Abstract (English)
Distantly Supervised Named Entity Recognition (DS-NER) has attracted attention due to its scalability and ability to automatically generate labeled data. However, distant annotation introduces many mislabeled instances, limiting its performance. Most of the existing work attempt to solve this problem by developing intricate models to learn from the noisy labels. An alternative approach is to attempt to clean the labeled data, thus increasing the quality of distant labels. This approach has received little attention for NER. In this paper, we propose a training dynamics-based label cleaning approach, which leverages the behavior of a model as training progresses to characterize the distantly annotated samples. We also introduce an automatic threshold estimation strategy to locate the errors in distant labels. Extensive experimental results demonstrate that: (1) models trained on our cleaned DS-NER datasets, which were refined by directly removing identified erroneous annotations, achieve significant improvements in F1-score, ranging from 3.18% to 8.95%; and (2) our method outperforms numerous advanced DS-NER approaches across four datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。