arXiv:2411.15585cs.CV2024-11NeurIPS被引 6

通过模拟人眼看与总结,提升复杂文本识别效果

Boosting Semi-Supervised Scene Text Recognition via Viewing and Summarizing

  • 用在线生成策略增强合成数据的字符样式多样性
  • 新损失函数使相同字符特征对齐,准确率达94.7%
  • 适合需要高精度文本识别的场景应用

现有场景文本识别方法在艺术化和严重扭曲文本上表现不佳,根源在于字符形态探索不足,包括合成数据单调、模型对形态敏感。受人类学习中‘看与总结’启发,我们提出无需人工标注的自驱对比学习框架,利用合成与真实未标注数据。在‘看’的过程中,设计在线生成策略,生成无背景干扰的多样化字符样本,使模型专注形态学习,提升对复杂样本的泛化能力。在‘总结’过程中,发现原有字符对比损失存在推导误差,导致类内分布稀疏并加剧难题样本歧义,因此提出新的字符单向对齐损失,将学生模型特征与教师模型参考特征对齐,统一相同字符的表示。大量实验表明,该方法在常见基准和Union14M-Benchmark上分别达到94.7%和70.9%的平均准确率,性能领先。代码将在https://github.com/qqqyd/ViSu公开。

原文摘要 · Abstract (English)

Existing scene text recognition (STR) methods struggle to recognize challenging texts, especially for artistic and severely distorted characters. The limitation lies in the insufficient exploration of character morphologies, including the monotonousness of widely used synthetic training data and the sensitivity of the model to character morphologies. To address these issues, inspired by the human learning process of viewing and summarizing, we facilitate the contrastive learning-based STR framework in a self-motivated manner by leveraging synthetic and real unlabeled data without any human cost. In the viewing process, to compensate for the simplicity of synthetic data and enrich character morphology diversity, we propose an Online Generation Strategy to generate background-free samples with diverse character styles. By excluding background noise distractions, the model is encouraged to focus on character morphology and generalize the ability to recognize complex samples when trained with only simple synthetic data. To boost the summarizing process, we theoretically demonstrate the derivation error in the previous character contrastive loss, which mistakenly causes the sparsity in the intra-class distribution and exacerbates ambiguity on challenging samples. Therefore, a new Character Unidirectional Alignment Loss is proposed to correct this error and unify the representation of the same characters in all samples by aligning the character features in the student model with the reference features in the teacher model. Extensive experiment results show that our method achieves SOTA performance (94.7\% and 70.9\% average accuracy on common benchmarks and Union14M-Benchmark). Code will be available at https://github.com/qqqyd/ViSu.

文本识别半监督对比学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。