一个轻量模型统一完成文档识别与定位,支持按提示查询文字位置。
PILOT: A Promptable Interleaved Layout-aware OCR Transformer
- 用提示词控制的生成式Transformer统一处理文字识别与坐标定位。
- 在多个数据集上达到竞品水平,参数仅155M比百亿模型小得多。
- 支持整页识别、区域阅读和关键词查找,适合交互式文档分析场景。
传统OCR将文档阅读拆分为检测、分割和识别三阶段,易受定位误差影响且难扩展至交互查询。本文提出PILOT,一个155M参数的提示条件生成模型,将文档OCR统一为序列生成任务。轻量深度可分离CNN编码页面,Transformer解码器自回归生成子词与10像素网格上的量化绝对坐标令牌流,实现整页识别、区域条件阅读及按字符串查询定位。通过从纯转录到联合文本-框生成再到提示控制提取的三阶段课程训练,稳定优化并提升空间定位能力。在IAM、RIMES~2009、SROIE~2019及异构MAURDOR基准上,PILOT在文字识别与行级检测上表现优于或媲美传统OCR系统、最新端到端HTR模型及紧凑视觉语言模型,同时显著小于百亿参数多模态模型。细粒度OCR与按字符串查询验证表明,统一文本-布局解码器可在紧凑架构中实现准确高效的提示式OCR。为保障可复现性,项目开源合成SROIE生成器、50万标注的IDL/PDFA页面、对齐的行级标注(IAM、RIMES~2009、MAURDOR)及源代码。
原文摘要 · Abstract (English)
Classical OCR pipelines decompose document reading into detection, segmentation, and recognition stages, which makes them sensitive to localization errors and difficult to extend to interactive querying. This work investigates whether a single compact model can jointly perform text recognition and spatial grounding on both handwritten and printed documents. We introduce PILOT, a 155M-parameter prompt-conditioned generative model that formulates document OCR as unified sequence generation. A lightweight depthwise-separable CNN encodes the page, and a Transformer decoder autoregressively emits a single stream of subword and quantized absolute-coordinate tokens on a 10\,px grid, enabling full-page OCR, region-conditioned reading, and query-by-string spotting within the same architecture. A three-stage curriculum, progressing from plain transcription to joint text-and-box generation and finally to prompt-controlled extraction, stabilizes training and improves spatial grounding. Experiments on IAM, RIMES~2009, SROIE~2019, and the heterogeneous MAURDOR benchmark show that PILOT achieves competitive or superior performance in text recognition and line-level detection compared with traditional OCR systems, recent end-to-end HTR models, and compact vision--language models, while remaining substantially smaller than billion-scale multimodal models. Additional evaluations on fine-grained OCR and query-by-string spotting further confirm that a unified text--layout decoder can provide accurate and efficient promptable OCR in a compact setting. To support reproducibility, we release the synthetic SROIE generator, the 500k annotated IDL/PDFA pages, the harmonized line-level annotations for IAM, RIMES~2009, and MAURDOR, and the source code at https://github.com/hamdilaziz/PILOT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。