统一处理印刷与手写文本行识别,支持多语言字符。
General Detection-based Text Line Recognition
- 用合成数据预训练+Transformer检测器,实现跨脚本字符定位。
- 在中文和密码文本上超越现有最佳性能,如CASIA v2和Borg数据集。
- 适合需要多语言通用识别方案的研究者与开发者。
我们提出一种通用的检测式文本行识别方法,适用于印刷体(OCR)与手写体(HTR),涵盖拉丁文、中文及密文字符。传统检测方法因字符级标注困难而较少用于HTR。本文通过三大关键洞察克服该问题:(i) 使用足够多样化的合成数据进行预训练,可学习任意书写系统的合理字符定位;(ii) 现代基于Transformer的检测器能并行检测大量实例,并在合适的掩码策略下利用不同检测结果间的一致性;(iii) 在具备粗略字符定位能力的预训练模型基础上,仅需线级标注即可在真实数据上微调,即使字母表不同也有效。所提方法DTLR采用完全不同于当前主流HTR的并行处理范式,而非逐字自回归生成。实验表明其在多种脚本上表现优异,尤其在中文字符识别(CASIA v2数据集)和密文识别(Borg与Copiale数据集)上优于现有方法。代码与模型已公开于https://github.com/raphael-baena/DTLR。
原文摘要 · Abstract (English)
We introduce a general detection-based approach to text line recognition, be it printed (OCR) or handwritten (HTR), with Latin, Chinese, or ciphered characters. Detection-based approaches have until now been largely discarded for HTR because reading characters separately is often challenging, and character-level annotation is difficult and expensive. We overcome these challenges thanks to three main insights: (i) synthetic pre-training with sufficiently diverse data enables learning reasonable character localization for any script; (ii) modern transformer-based detectors can jointly detect a large number of instances, and, if trained with an adequate masking strategy, leverage consistency between the different detections; (iii) once a pre-trained detection model with approximate character localization is available, it is possible to fine-tune it with line-level annotation on real data, even with a different alphabet. Our approach, dubbed DTLR, builds on a completely different paradigm than state-of-the-art HTR methods, which rely on autoregressive decoding, predicting character values one by one, while we treat a complete line in parallel. Remarkably, we demonstrate good performance on a large range of scripts, usually tackled with specialized approaches. In particular, we improve state-of-the-art performances for Chinese script recognition on the CASIA v2 dataset, and for cipher recognition on the Borg and Copiale datasets. Our code and models are available at https://github.com/raphael-baena/DTLR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。