融合卷积与文本信息，提升手写文字识别精度与泛化能力
HTR-ConvText: Leveraging Convolution and Textual Information for Handwritten Text Recognition
- 采用残差卷积+MobileViT结构捕捉笔画级局部特征
- 在IAM等4个数据集上超越现有方法，小样本下表现更优
- 适合书写风格多样、训练数据少的手写识别场景
手写文字识别因数据有限、书写风格差异大及复杂变音符号而面临挑战。现有方法虽部分缓解问题，但在缺乏大量合成数据时泛化能力不足。本文提出HTR-ConvText模型，通过残差卷积神经网络与带位置编码的MobileViT模块结合，同时捕获结构模式与细微书写特征。进一步设计混合式ConvText编码器，在分层结构中融合全局上下文与局部特征，降低序列长度以提升效率。此外，引入辅助模块注入文本上下文，增强连接时序分类（CTC）的鲁棒性。在IAM、READ2016、LAM和HANDS-VNOnDB数据集上的实验表明，该方法在有限训练样本和高书写多样性场景下均实现更高性能与更好泛化能力。
原文摘要 · Abstract (English)
Handwritten Text Recognition remains challenging due to the limited data, high writing style variance, and scripts with complex diacritics. Existing approaches, though partially address these issues, often struggle to generalize without massive synthetic data. To address these challenges, we propose HTR-ConvText, a model designed to capture fine-grained, stroke-level local features while preserving global contextual dependencies. In the feature extraction stage, we integrate a residual Convolutional Neural Network backbone with a MobileViT with Positional Encoding block. This enables the model to both capture structural patterns and learn subtle writing details. We then introduce the ConvText encoder, a hybrid architecture combining global context and local features within a hierarchical structure that reduces sequence length for improved efficiency. Additionally, an auxiliary module injects textual context to mitigate the weakness of Connectionist Temporal Classification. Evaluations on IAM, READ2016, LAM and HANDS-VNOnDB demonstrate that our approach achieves improved performance and better generalization compared to existing methods, especially in scenarios with limited training samples and high handwriting diversity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。