针对大模型标注噪声，提出按错误类型分别加权损失，提升命名实体识别鲁棒性。
Error-Type-Aware Loss Reweighting for Robust Named Entity Recognition with Noisy LLM Labels
- 区分缺失提及和类型错误，设计不同重加权策略
- 在15%-40%噪声下，平均提升F1 0.8-2.0个百分点
- 无需额外训练资源，适合用大模型生成标签的场景
大语言模型被广泛用于为小型任务模型（如命名实体识别）生成标注数据。然而，现有方法假设合成数据无误，实际中大模型引入的标注噪声会损害性能。我们发现：当前微调忽略噪声导致性能下降；已有抗噪损失不适用于序列标注，因命名实体识别中的噪声具有异质性——例如遗漏提及与类型错误对训练信号影响不同。若对所有错误标记统一处理，可能消除有效监督或强化错误标签。为此，我们提出面向错误类型的损失重加权方法，对不同错误类型采用独立重加权规则。该方法简单高效，无需额外训练资源，在噪声率15%至40%时，数据集平均提升F1 0.8–2.0个百分点，最高达4.6个百分点（在Wikigold上，噪声率为24.1%）。
原文摘要 · Abstract (English)
Large language models are increasingly used to annotate datasets for training smaller, task-specialized models such as named entity recognition. While this method yields effective models, it assumes that the synthetic dataset is correctly annotated. In this work, we find that (i) current fine-tuning processes simply ignore LLM-introduced annotation noise, resulting in degraded performance and (ii) existing noise-robust losses are not transferable to sequence labeling because annotation noise in named entity recognition is heterogeneous: for example, missing mentions and type errors affect the training signal in different ways. Treating all noisy tokens equally in noise-robust losses and applying a single reweighing criterion for all may therefore remove useful supervision or reinforce incorrect labels. To address this limitation, we propose error-type-aware loss reweighting for NER, which introduces separate reweighing rules for different types of potentially erroneous tokens. Our approach is simple and efficient, does not require additional training resources, and improves F1 by 0.8 - 2.0 percentage points on dataset-level average for noise levels between 15% and 40%, with a maximum improvement of 4.6 percentage points with 24.1% noise on Wikigold.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。