调优TrOCR识别中世纪手稿，发现冻结层数与预处理对准确率影响显著。
TrOCR for Medieval HTR: A Systematic Ablation Study with Cross-Dataset Validation
- 控制性实验研究三种微调策略对性能的影响。
- 最佳配置达8.03%字符错误率，去除对比度归一化仍可保持高精度。
- 揭示模型失败模式，适合历史文献数字化研究者参考。
将基于Transformer的手写文本识别（HTR）模型微调至中世纪手稿面临挑战，因这些模型在现代文本上预训练，需适应截然不同的视觉域。本文系统研究了三种可控微调选择（对比度归一化、数据增强、层冻结）在将TrOCR适配小规模历史数据集时对识别准确率的影响。在13世纪意大利手稿I-CT 91（“Cortonese”）上进行受控实验，并在公开的READ-16基准上复现相同实验网格以验证鲁棒性。在Cortonese上，最优配置实现8.03%字符错误率（CER）。13种配置的统计比较显示，冻结最多三编码器层或六解码器层不会显著损害准确率，但更深冻结会逐步恶化性能。移除对比度归一化（CLAHE）后得到7.84% CER，与领域专用基线相当，表明强优化可降低对图像预处理的依赖。跨数据集验证在READ-16上显示，解码器冻结阈值比编码器更具可迁移性，而联合冻结策略需针对特定数据集重新验证。最后，利用Grad-CAM梯度归因和解码器交叉注意力图诊断误判模式与失败机制。源代码已开源于https://github.com/LaudareProject/TrOCR-analysis。
原文摘要 · Abstract (English)
Fine-tuning transformer-based handwritten text recognition (HTR) models on medieval manuscripts is challenging because these models are pre-trained on modern text and must adapt to a very different visual domain. This paper studies how three controllable fine-tuning choices (contrast normalization, data augmentation, and layer freezing) affect recognition accuracy when adapting TrOCR to small historical datasets. We run controlled experiments on a 13th-century Italian manuscript (I-CT 91 "Cortonese") and replicate the same experimental grid on the public READ-16 benchmark as robustness evidence. On Cortonese, our best configuration achieves 8.03% character error rate (CER). Statistical comparisons across 13 configurations show that freezing up to three encoder layers or six decoder layers does not significantly harm accuracy, while deeper freezing becomes progressively detrimental. Removing contrast normalization (CLAHE) yields 7.84% CER, comparable to a domain-specialized baseline, suggesting strong optimization can reduce reliance on image preprocessing. Cross-dataset validation on READ-16 shows that decoder freezing thresholds transfer more robustly than encoder thresholds, and combined freezing strategies require dataset-specific re-validation. Finally, we use Grad-CAM gradient attributions and decoder cross-attention maps to diagnose error patterns and failure modes revealed by the ablations. Source code is available at https://github.com/LaudareProject/TrOCR-analysis
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。