测试100多个文字的OCR通用性，发现主流模型仅在少数语言上表现好。
GlotOCR Bench: OCR Models Still Struggle Beyond a Handful of Unicode Scripts
- 构建覆盖100+文字的OCR评估基准，含清晰与退化图像。
- 多数模型仅在不到10个文字上表现良好，最强模型也难超30个。
- 性能依赖预训练覆盖度，陌生文字常误认或胡编字符。
光学字符识别（OCR）随视觉语言模型发展迅速，但评估仍集中于少数高/中资源文字。我们提出GlotOCR Bench，一个涵盖100多个Unicode文字的全面评估基准。该基准基于真实多语言文本生成清晰与退化图像，使用Google Fonts字体、HarfBuzz排版、FreeType渲染，支持从左到右与从右到左文字。所有渲染样本经人工审核确保正确性。我们评估了广泛开源与专有视觉语言模型，发现多数模型仅在少于十个文字上表现良好，即使顶尖模型也无法在超过三十个文字上有效泛化。性能普遍与文字级预训练覆盖度相关，表明当前OCR系统更依赖语言模型预训练而非视觉识别。面对陌生文字，模型要么输出随机噪声，要么虚构相似已知文字的字符。我们公开基准与流水线以保证可复现性。代码：https://github.com/cisnlp/glotocr-bench，数据集：https://hf.co/datasets/cis-lmu/glotocr-bench。
原文摘要 · Abstract (English)
Optical character recognition (OCR) has advanced rapidly with the rise of vision-language models, yet evaluation has remained concentrated on a small cluster of high- and mid-resource scripts. We introduce GlotOCR Bench, a comprehensive benchmark evaluating OCR generalization across 100+ Unicode scripts. Our benchmark comprises clean and degraded image variants rendered from real multilingual texts. Images are rendered using fonts from the Google Fonts repository, shaped with HarfBuzz and rasterized with FreeType, supporting both LTR and RTL scripts. Samples of rendered images were manually reviewed to verify correct rendering across all scripts. We evaluate a broad suite of open-weight and proprietary vision-language models and find that most perform well on fewer than ten scripts, and even the strongest frontier models fail to generalize beyond thirty scripts. Performance broadly tracks script-level pretraining coverage, suggesting that current OCR systems rely on language model pretraining as much as on visual recognition. Models confronted with unfamiliar scripts either produce random noise or hallucinate characters from similar scripts they already know. We release the benchmark and pipeline for reproducibility. Pipeline Code: https://github.com/cisnlp/glotocr-bench, Benchmark: https://hf.co/datasets/cis-lmu/glotocr-bench.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。