通过跨尺度预测提升无监督场景文字识别性能
Masked Next-Scale Prediction for Self-supervised Scene Text Recognition
- 设计跨尺度预测机制，学习从低分辨率到高分辨率的文本结构演化
- 在Union14M上达86.2%平均准确率，六数据集综合96.7%表现
- 适合关注自监督视觉建模与文本结构理解的研究者
场景文字识别需建模从粗略布局到细粒度字符笔画的视觉结构演变。现有自监督方法如掩码图像建模（MIM）虽缓解标注依赖，但大多仅在单一空间尺度操作，难以捕捉文本的层级特性。本文提出掩码下一尺度预测（MNSP），一种统一的自监督框架，显式建模跨尺度结构演化。该框架引入下一尺度预测（NSP），通过低分辨率上下文预测高分辨率特征以学习层次化表示。然而，朴素尺度预测易导致空间注意力发散，偏向背景区域。MNSP通过联合学习跨尺度预测与掩码图像重建解决此问题：NSP捕捉跨分辨率的全局布局先验，而掩码重建施加强局部约束，引导注意力聚焦于文本区域。此外，多尺度语言对齐模块维持不同分辨率间的语义一致性。大量实验表明，MNSP在挑战性Union14M基准上达到86.2%平均准确率，并在六个标准数据集上实现96.7%综合性能。额外分析显示，该方法在极端尺度与布局变化下仍具更强鲁棒性。代码已开源。
原文摘要 · Abstract (English)
Scene Text Recognition requires modeling visual structures that evolve from coarse layouts to fine-grained character strokes. Training such models relies on large amounts of annotated data. Recent self-supervised approaches, such as Masked Image Modeling (MIM), alleviate this dependency by leveraging large-scale unlabeled data. Yet most existing MIM methods operate at a single spatial scale and fail to capture the hierarchical nature of scene text. In this work, we introduce Masked Next-Scale Prediction (MNSP), a unified self-supervised framework designed to explicitly model cross-scale structural evolution. The framework incorporates Next-Scale Prediction (NSP), which learns hierarchical representations by predicting higher-resolution features from lower-resolution contexts. Naive scale prediction, however, tends to produce spatially diffuse attention, directing the model toward background regions rather than textual structures. MNSP resolves this limitation by jointly learning cross-scale prediction and masked image reconstruction. NSP captures global layout priors across resolutions, while masked reconstruction imposes strong local constraints that guide attention toward informative text regions. A Multi-scale Linguistic Alignment module further maintains semantic consistency across different resolutions. Extensive experiments demonstrate that MNSP achieves state-of-the-art performance, reaching 86.2\% average accuracy on the challenging Union14M benchmark and 96.7\% across six standard datasets. Additional analyses show that our method improves robustness under extreme scale and layout variations. Code is available at https://github.com/CzhczhcHczh/MNSP
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。