通过解码器扩展与自蒸馏提升文本识别精度
Accurate Scene Text Recognition with Efficient Model Scaling and Cloze Self-Distillation
- 解码器放大比编码器更有效，显著提升识别性能
- 在11个基准中10个达顶尖水平，参数和计算量更低
- 用上下文软预测缓解真实数据标签噪声问题
模型扩展已被证实可提升场景文本识别（STR）性能，但视觉编码器与文本解码器各自扩展的贡献仍不明确。本文通过实证分析发现，与以往认知相反，解码器扩展带来的性能提升始终超过仅扩展编码器的效果。我们还指出，真实数据中的标签噪声是限制STR模型表现的关键挑战。为此，提出闭塞自蒸馏（CSD）方法，通过教师模型生成的上下文感知软预测和伪标签来训练学生模型，以减轻标签噪声影响。此外，引入针对STR的差异化交叉注意力机制优化解码器结构。所提方法仅使用真实数据，在11个基准中的10个上达到最先进性能，同时大幅降低参数量与计算开销。
原文摘要 · Abstract (English)
Scaling architectures have been proven effective for improving Scene Text Recognition (STR), but the individual contribution of vision encoder and text decoder scaling remain under-explored. In this work, we present an in-depth empirical analysis and demonstrate that, contrary to previous observations, scaling the decoder yields significant performance gains, always exceeding those achieved by encoder scaling alone. We also identify label noise as a key challenge in STR, particularly in real-world data, which can limit the effectiveness of STR models. To address this, we propose Cloze Self-Distillation (CSD), a method that mitigates label noise by distilling a student model from context-aware soft predictions and pseudolabels generated by a teacher model. Additionally, we enhance the decoder architecture by introducing differential cross-attention for STR. Our methodology achieves state-of-the-art performance on 10 out of 11 benchmarks using only real data, while significantly reducing the parameter size and computational costs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。