arXiv:2502.07172cs.CVcs.AI2025-02被引 2

用伪标签提升手写公式识别,仅需少量标注数据即可大幅提高准确率。

SemiHMER: Semi-supervised Handwritten Mathematical Expression Recognition using pseudo-labels

  • 双分支一致性正则化,通过伪标签约束两个网络输出一致。
  • 在CROHME14/16/19上平均准确率提升5.47%/4.87%/5.25%。
  • 新增全局动态计数模块,有效减少长公式识别错误和重复字符。

本文研究通过利用标注数据与额外未标注数据,实现半监督手写数学表达式识别。提出一种名为SemiHMER的新颖一致性正则化框架,采用双分支半监督学习机制:对同一输入图像,一个扰动网络生成伪标签,用于监督另一个网络,使用标准交叉熵损失。该一致性策略促使两个扰动网络对同一输入的预测高度相似,并通过伪标签扩展训练数据。进一步引入弱增强到强增强策略,在不同分支施加不同强度的增广,有效扩充训练数据并提升模型训练质量。此外,提出全局动态计数模块(GDCM),缓解长距离公式识别中的误判问题,降低重复字符出现频率。实验表明,相比基线模型,SemiHMER在CROHME14、CROHME16、CROHME19上的平均准确率分别提升5.47%、4.87%和5.25%。

原文摘要 · Abstract (English)

In this paper, we study semi-supervised Handwritten Mathematical Expression Recognition (HMER) via exploring both labeled data and extra unlabeled data. We propose a novel consistency regularization framework, termed SemiHMER, which introduces dual-branch semi-supervised learning. Specifically, we enforce consistency between the two networks for the same input image. The pseudo-label, generated by one perturbed recognition network, is utilized to supervise the other network using the standard cross-entropy loss. The SemiHMER consistency encourages high similarity between the predictions of the two perturbed networks for the same input image and expands the training data by leveraging unlabeled data with pseudo-labels. We further introduce a weak-to-strong strategy by applying different levels of augmentation to each branch, effectively expanding the training data and enhancing the quality of network training. Additionally, we propose a novel module, the Global Dynamic Counting Module (GDCM), to enhance the performance of the HMER decoder by alleviating recognition inaccuracies in long-distance formula recognition and reducing the occurrence of repeated characters. The experimental results demonstrate that our work achieves significant performance improvements, with an average accuracy increase of 5.47% on CROHME14, 4.87% on CROHME16, and 5.25% on CROHME19, compared to our baselines.

手写识别半监督数学表达式伪标签

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。