arXiv:2412.18981cs.CVcs.LG2024-12被引 2

端到端识别手写文档文字与版面,精度超现有方法。

HAND: Hierarchical Attention Network for Multi-Scale Handwritten Document Recognition and Layout Analysis

  • 分层注意力架构同步处理文字识别与版面分析
  • 在READ 2016上字符错误率降低59.8%(行级)
  • 适合复杂古籍等多尺度手写文档处理

手写文档识别(HDR)因书写风格多样、版面复杂,在计算机视觉中极具挑战。传统方法将文本识别与版面分析割裂处理,难以有效融合。本文提出HAND(分层注意力网络),一种端到端、无需分割的多尺度手写文档识别与版面分析模型。其核心包括:融合门控深度可分离卷积与八度卷积的先进卷积编码器,用于鲁棒特征提取;动态适应文档复杂度的多尺度自适应处理(MSAP)框架;以及带有记忆增强与稀疏注意力机制的分层解码器。该架构可有效应对从单行到三栏页面的多尺度输入,同时保持计算高效。此外,模型采用五级渐进式学习策略,并通过领域自适应预训练mT5模型对复杂古籍进行后处理优化。在READ 2016数据集上的大量实验表明,相比现有最优方法,HAND在行级识别上实现59.8%的字符错误率(CER)下降,页级达31.2%。模型参数量仅5.60M,同时刷新了文本识别与版面分析双项基准。源代码与预训练模型已开源。

原文摘要 · Abstract (English)

Handwritten document recognition (HDR) is one of the most challenging tasks in the field of computer vision, due to the various writing styles and complex layouts inherent in handwritten texts. Traditionally, this problem has been approached as two separate tasks, handwritten text recognition and layout analysis, and struggled to integrate the two processes effectively. This paper introduces HAND (Hierarchical Attention Network for Multi-Scale Document), a novel end-to-end and segmentation-free architecture for simultaneous text recognition and layout analysis tasks. Our model's key components include an advanced convolutional encoder integrating Gated Depth-wise Separable and Octave Convolutions for robust feature extraction, a Multi-Scale Adaptive Processing (MSAP) framework that dynamically adjusts to document complexity and a hierarchical attention decoder with memory-augmented and sparse attention mechanisms. These components enable our model to scale effectively from single-line to triple-column pages while maintaining computational efficiency. Additionally, HAND adopts curriculum learning across five complexity levels. To improve the recognition accuracy of complex ancient manuscripts, we fine-tune and integrate a Domain-Adaptive Pre-trained mT5 model for post-processing refinement. Extensive evaluations on the READ 2016 dataset demonstrate the superior performance of HAND, achieving up to 59.8% reduction in CER for line-level recognition and 31.2% for page-level recognition compared to state-of-the-art methods. The model also maintains a compact size of 5.60M parameters while establishing new benchmarks in both text recognition and layout analysis. Source code and pre-trained models are available at : https://github.com/MHHamdan/HAND.

手写识别版面分析注意力机制多尺度

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。