针对古籍文字识别难题,提出兼顾版式与文本增强的多模态模型。
TongGuOCR: A Layout-Aware and Token-Augmented OCR MLLM for Chinese Historical Documents

- 通过版式感知预处理构建连贯识别块,减少区域干扰。
- 字符级词汇扩展+行间位移标记,提升罕见字识别与复杂阅读路径建模能力。
- 在古籍数据集上显著优于主流OCR模型,适合古籍数字化项目使用。
中文历史文献承载宝贵文化遗产,但许多资料仅以扫描图像形式存在,难以实现全文检索、校勘与计算分析。光学字符识别(OCR)可弥补这一缺口,但因古籍常含复杂版式、生僻字及非线性阅读顺序,准确识别仍具挑战。本文提出TongGuOCR,一种面向中文古籍的版式感知与令牌增强型多模态大模型。首先,版式感知预处理模块构建并优化局部连贯的识别块,保留局部上下文同时降低跨区域干扰。其次,令牌增强识别模块在两个互补层面增强转录目标:字符级词汇扩展为每个罕见字赋予单一令牌表示,缩短解码路径;行间过渡建模引入离散空间位移令牌,引导解码器沿复杂阅读路径前行,无需精确坐标。在两个中文古籍OCR基准测试中,TongGuOCR超越代表性专用OCR模型、通用多模态大模型及面向OCR的MLLM。在更具挑战性的M5HisDoc基准上,达到93.76的AR,将NED从10.43降至6.15,RO-ED从7.53降至3.49,优于现有最佳结果。在线演示地址:https://jzzh2004.github.io/TongGuOCR。
原文摘要 · Abstract (English)
Chinese historical documents preserve valuable cultural heritage, but many collections remain accessible only as scanned page images, preventing full-text retrieval, collation, and computational analysis. Optical character recognition (OCR) can bridge this gap, but accurate transcription remains challenging because historical documents often contain complex layouts, rare characters, and nontrivial reading orders. We propose TongGuOCR, a layout-aware and token-augmented multimodal large language model (MLLM) for OCR of Chinese historical documents. First, a Layout-Aware Preprocessing module constructs and refines locally coherent recognition blocks to preserve local context while reducing interference across regions. Second, a Token-Augmented Recognition module augments the transcription target at two complementary levels: character-level vocabulary expansion gives each rare glyph a direct one-token representation and shortens its decoding path, while line-to-line transition modeling injects discrete spatial displacement tokens that guide the decoder along complex reading paths without requiring precise coordinates. Experiments on two Chinese historical document OCR benchmarks show that TongGuOCR outperforms representative traditional task-specific OCR models, general-purpose MLLMs, and OCR-oriented MLLMs. On the more challenging M5HisDoc benchmark, TongGuOCR achieves 93.76 AR and reduces NED from 10.43 to 6.15 and RO-ED from 7.53 to 3.49 relative to the best competing score for each metric. An online demo is available at https://jzzh2004.github.io/TongGuOCR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。