高效识别历史手写文本，融合注意力与知识蒸馏提升准确率。
HTR-JAND: Handwritten Text Recognition with Joint Attention Network and Knowledge Distillation
- 用联合注意力机制融合自注意力与近邻注意力，增强序列建模能力。
- 在IAM、RIMES、Bentham数据集上达到1.23%、1.02%、2.02%的最低字符错误率。
- 模型参数减少48%仍保持高精度，适合资源受限场景使用。
尽管深度学习取得显著进展，当前手写文本识别（HTR）系统在处理历史文档时仍面临书写风格多样、文本质量退化及跨语言多时期计算效率等挑战。本文提出HTR-JAND框架，结合联合注意力网络与知识蒸馏，实现高效特征提取与模型压缩。其核心包含三部分：(1) 融合FullGatedConv2d与挤压-激励模块的CNN架构，实现自适应特征提取；(2) 联合注意力机制，融合多头自注意力与近邻注意力，增强序列建模鲁棒性；(3) 基于课程学习的知识蒸馏框架，实现高效压缩并保留精度。采用多阶段训练策略，包括课程学习、合成数据生成与多任务学习，支持跨数据集知识迁移。通过上下文感知的T5后处理进一步提升历史文档识别准确率。实验表明，HTR-JAND在IAM、RIMES和Bentham数据集上分别取得1.23%、1.02%、2.02%的最优字符错误率（CER）。学生模型参数量从1.5M降至0.75M（减少48%），性能仍具竞争力。代码与预训练模型已开源。
原文摘要 · Abstract (English)
Despite significant advances in deep learning, current Handwritten Text Recognition (HTR) systems struggle with the inherent complexity of historical documents, including diverse writing styles, degraded text quality, and computational efficiency requirements across multiple languages and time periods. This paper introduces HTR-JAND (HTR-JAND: Handwritten Text Recognition with Joint Attention Network and Knowledge Distillation), an efficient HTR framework that combines advanced feature extraction with knowledge distillation. Our architecture incorporates three key components: (1) a CNN architecture integrating FullGatedConv2d layers with Squeeze-and-Excitation blocks for adaptive feature extraction, (2) a Combined Attention mechanism fusing Multi-Head Self-Attention with Proxima Attention for robust sequence modeling, and (3) a Knowledge Distillation framework enabling efficient model compression while preserving accuracy through curriculum-based training. The HTR-JAND framework implements a multi-stage training approach combining curriculum learning, synthetic data generation, and multi-task learning for cross-dataset knowledge transfer. We enhance recognition accuracy through context-aware T5 post-processing, particularly effective for historical documents. Comprehensive evaluations demonstrate HTR-JAND's effectiveness, achieving state-of-the-art Character Error Rates (CER) of 1.23\%, 1.02\%, and 2.02\% on IAM, RIMES, and Bentham datasets respectively. Our Student model achieves a 48\% parameter reduction (0.75M versus 1.5M parameters) while maintaining competitive performance through efficient knowledge transfer. Source code and pre-trained models are available at \href{https://github.com/DocumentRecognitionModels/HTR-JAND}{Github}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。