用3D注意力与多尺度训练提升手写文字识别准确率
Improving Handwritten Text Recognition via 3D Attention and Multi-Scale Training
- 设计3D注意力模块，融合全局与局部上下文信息
- 在中英文手写数据集上达到顶尖水平，准确率超90%
- 适合做手写识别研究或实际应用的开发者参考
针对无分割手写文本识别，现有方法可分为三类：连接时序分类（CTC）、隐马尔可夫模型和编码器-解码器。本文受这三类方法启发，提出一种新网络架构，引入三维（3D）注意力模块和全局-局部上下文信息。基于最后一层卷积特征图，生成不同分辨率的3D块，输入3D注意力模块以提取序列化视觉特征。最终通过融合视觉特征与对应上下文特征，获得优化表示。网络整合注意力、全连接层、循环单元和卷积层，通过CTC损失与交叉熵损失联合训练。在最新的中文手写数据集SCUT-HCCDoc和SCUT-EPT，以及英文数据集IAM上的实验表明，该方法性能媲美当前最优，代码已开源。
原文摘要 · Abstract (English)
The segmentation-free research efforts for addressing handwritten text recognition can be divided into three categories: connectionist temporal classification (CTC), hidden Markov model and encoder-decoder methods. In this paper, inspired by the above three modeling methods, we propose a new recognition network by using a novel three-dimensional (3D) attention module and global-local context information. Based on the feature maps of the last convolutional layer, a series of 3D blocks with different resolutions are split. Then, these 3D blocks are fed into the 3D attention module to generate sequential visual features. Finally, by fusing the visual features and the corresponding global-local context features, a well-designed representation can be obtained. Main canonical neural units including attention mechanisms, fully-connected layers, recurrent units and convolutional layers are efficiently organized into a network and can be jointly trained by the CTC loss and the cross-entropy loss. Experiments on the latest Chinese handwritten text datasets (the SCUT-HCCDoc and the SCUT-EPT) and one English handwritten text dataset (the IAM) show that the proposed method can achieve comparable results with the state-of-the-art methods. The code is available at https://github.com/Wukong90/3DAttention-MultiScaleTraining-for-HTR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。