用半监督方法提升任意形状文字检测识别,仅用少量标注数据就超越强监督模型。
SemiETS: Integrating Spatial and Content Consistencies for Semi-Supervised End-to-end Text Spotting
- 分层生成检测与识别任务的伪标签,减少噪声。
- 在0.5%标注数据下,总文本集上H均值提升8.7%。
- 适用于多种文字检测器,适合实际场景部署。
现有场景文字检测方法依赖高质量人工标注,成本高昂。为降低标注开销,本文研究半监督文字检测(SSTS),利用未标注图像中的信息。但直接套用通用半监督方法会面临两大挑战:检测与识别任务间伪标签不一致、师生模型间监督信号不一致。为此提出端到端半监督文字检测框架SemiETS,充分利用检测与识别任务的互补性。通过渐进式生成可靠的分层伪标签,降低噪声;同时设计双向信息流,提取位置与文本内容的关键特征以增强一致性。在三个数据集上大量实验表明,SemiETS在任意形状文字检测上表现优异。例如,在总文本集上,仅使用0.5%、1%、2%标注数据时,其端到端检测的H均值分别比现有最佳半监督方法高出8.7%、5.6%和2.6%。更关键的是,其性能仍优于使用大量标注数据训练的强监督检测器2.0%。该方法展现出强大领域适应能力,具有实际应用潜力。此外,其优势在不同文字检测器上均保持一致。
原文摘要 · Abstract (English)
Most previous scene text spotting methods rely on high-quality manual annotations to achieve promising performance. To reduce their expensive costs, we study semi-supervised text spotting (SSTS) to exploit useful information from unlabeled images. However, directly applying existing semi-supervised methods of general scenes to SSTS will face new challenges: 1) inconsistent pseudo labels between detection and recognition tasks, and 2) sub-optimal supervisions caused by inconsistency between teacher/student. Thus, we propose a new Semi-supervised framework for End-to-end Text Spotting, namely SemiETS that leverages the complementarity of text detection and recognition. Specifically, it gradually generates reliable hierarchical pseudo labels for each task, thereby reducing noisy labels. Meanwhile, it extracts important information in locations and transcriptions from bidirectional flows to improve consistency. Extensive experiments on three datasets under various settings demonstrate the effectiveness of SemiETS on arbitrary-shaped text. For example, it outperforms previous state-of-the-art SSL methods by a large margin on end-to-end spotting (+8.7%, +5.6%, and +2.6% H-mean under 0.5%, 1%, and 2% labeled data settings on Total-Text, respectively). More importantly, it still improves upon a strongly supervised text spotter trained with plenty of labeled data by 2.0%. Compelling domain adaptation ability shows practical potential. Moreover, our method demonstrates consistent improvement on different text spotters.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。