让手写公式识别更透明,可逐项分析错误。
The Return of Structural Handwritten Mathematical Expression Recognition
- 用神经网络自动生成符号与笔画的对应标注
- 在CROHME-2023上达到领先性能,支持空间关系建模
- 输出带完整结构图,适合需要可解释性的教育应用
手写数学表达式识别是教育技术的基础,支持数字笔记和自动评分等应用。尽管现代编码器-解码器架构结合大语言模型在生成LaTeX方面表现优异,但缺乏符号与笔画间的显式对齐,限制了错误分析、可解释性以及需空间感知的交互式更新。本文提出一种结构化识别方法,包含两项创新:1)基于神经网络的自动标注系统,将LaTeX方程映射至原始笔迹,自动生成符号分割、分类和空间关系标注;2)模块化结构识别系统,独立优化分割、分类与关系预测。利用我们自标注系统构建的结构化数据集,所提方法结合图结构笔迹排序、混合卷积-循环网络与Transformer修正,在CROHME-2023基准上取得竞争力表现。关键优势在于生成完整图结构,直接关联笔迹与预测符号,实现透明错误分析与可解释输出。
原文摘要 · Abstract (English)
Handwritten Mathematical Expression Recognition is foundational for educational technologies, enabling applications like digital note-taking and automated grading. While modern encoder-decoder architectures with large language models excel at LaTeX generation, they lack explicit symbol-to-trace alignment, a critical limitation for error analysis, interpretability, and spatially aware interactive applications requiring selective content updates. This paper introduces a structural recognition approach with two innovations: 1 an automatic annotation system that uses a neural network to map LaTeX equations to raw traces, automatically generating annotations for symbol segmentation, classification, and spatial relations, and 2 a modular structural recognition system that independently optimizes segmentation, classification, and relation prediction. By leveraging a dataset enriched with structural annotations from our auto-labeling system, the proposed recognition system combines graph-based trace sorting, a hybrid convolutional-recurrent network, and transformer-based correction to achieve competitive performance on the CROHME-2023 benchmark. Crucially, our structural recognition system generates a complete graph structure that directly links handwritten traces to predicted symbols, enabling transparent error analysis and interpretable outputs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。