针对历史职教文档的识别噪声,提出抗噪实体识别方法。
Noise-Aware Named Entity Recognition for Historical VET Documents
- 用合成错误数据训练,让模型学会识别扫描噪声。
- 在德语文档上准确率提升显著,多类型实体可识别。
- 适合历史文献数字化、档案整理等场景使用。
本文聚焦职业培训(VET)领域的历史数字化文档中的命名实体识别(NER)问题,这些文档常因光学字符识别(OCR)产生噪声。提出一种鲁棒的NER方法,结合噪声感知训练(NAT)、迁移学习和多阶段微调。系统比较了在噪声数据、干净数据及人工合成数据上训练的三种策略。该方法是首批能识别多类实体的VET文档NER方案之一,虽以德语文档为实验对象,但具备跨语言可迁移性。实验证明,领域特定且噪声感知的微调能显著提升噪声环境下的鲁棒性与准确率。代码已公开,支持领域特定场景下的可复现噪声感知NER。
原文摘要 · Abstract (English)
This paper addresses Named Entity Recognition (NER) in the domain of Vocational Education and Training (VET), focusing on historical, digitized documents that suffer from OCR-induced noise. We propose a robust NER approach leveraging Noise-Aware Training (NAT) with synthetically injected OCR errors, transfer learning, and multi-stage fine-tuning. Three complementary strategies, training on noisy, clean, and artificial data, are systematically compared. Our method is one of the first to recognize multiple entity types in VET documents. It is applied to German documents but transferable to arbitrary languages. Experimental results demonstrate that domain-specific and noise-aware fine-tuning substantially increases robustness and accuracy under noisy conditions. We provide publicly available code for reproducible noise-aware NER in domain-specific contexts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。