跨语言手写识别提升源于序列建模,而非仅依赖视觉相似性。
Understanding Cross-Language Transfer Improvements in Low-Resource HTR: The Role of Sequence Modeling
- 对比纯卷积与带序列建模的模型,控制训练条件验证迁移机制。
- 在100、500、1000样本低资源下,CRNN模型跨语言性能显著更优。
- 适合关注低资源场景下跨语言迁移的NLP与HTR研究者阅读。
阿拉伯语系手写文本识别(HTR)在低资源条件下可通过跨语言联合训练获益,尤其使用结合卷积编码器与序列建模的CRNN模型。然而,这种提升究竟源于共享视觉表征还是序列级依赖尚不明确。本文在相同单语与多语训练设置下,对阿拉伯(KHATT)、乌尔都(NUST-UHWR)、波斯(PHTD)三数据集进行线级阿拉伯语系HTR的受控架构研究,比较仅含CNN的模型与采用CTC解码的CRNN模型。实验在低资源设定(K ∈ {100, 500, 1000})下进行。结果表明:仅用CNN的模型跨语言提升有限且不稳定,而CRNN模型在多语言训练中表现更优,尤其在数据最稀缺情况下。聚焦于迁移提升(ΔCER),发现跨语言性能增益与序列建模相关,而仅靠卷积编码器学习到的视觉表征(如字符形状相似性)不足以实现有效迁移。这表明上下文建模在低资源跨语言迁移中起关键作用,该现象或可推广至其他低资源语言场景。
原文摘要 · Abstract (English)
Handwritten Text Recognition (HTR) for Arabic-script languages benefits from cross-language joint training under low-resource conditions, particularly when using CRNN-based models that combine convolutional encoders with sequence modeling. However, it remains unclear whether these improvements are better explained by shared visual representations or sequence-level dependencies. In this work, we conduct a controlled architectural study of line-level Arabic-script HTR, comparing CNN-only models with CTC decoding and CRNN models under identical single-script and multi-script training regimes. Experiments are performed on Arabic (KHATT), Urdu (NUST-UHWR), and Persian (PHTD) datasets under low-resource settings (K in {100, 500, 1000}). Our results show a clear divergence in transfer behavior: while CNN-only models exhibit limited or unstable improvements, CRNN models achieve better performance under multi-script training, particularly in the most data-constrained regimes. Focusing on transfer improvements (delta CER) rather than absolute performance, we find that cross-language improvements are associated with sequence-level modeling, while sharing visual representations learned by the CNN encoder, corresponding to similarities in character shapes across scripts, alone appears to be insufficient. This finding suggests that contextual modeling plays an important role in enabling effective transfer in low-resource scenarios, and that similar behavior may extend to other low-resource language settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。