视觉语言模型能读文字却看不懂字体，这篇论文诊断并尝试解决这一问题。
Reading $\neq$ Seeing: Diagnosing and Closing the Typography Gap in Vision-Language Models
- 构建跨26种字体的评估框架，系统测试模型对字体风格、大小、颜色的识别能力
- 发现模型对颜色识别近乎完美，但字体风格识别普遍极差，且模型规模不影响表现
- 小样本微调可显著提升字体大小识别，但字体风格仍需架构创新
视觉语言模型在图像文字识别上已达近完美准确率，却普遍存在排版盲区：能理解文字内容，却无法感知其外观。我们通过在26种字体、4种文字体系和3个难度等级下评估15个最先进的视觉语言模型，系统诊断该差距。结果显示，颜色识别接近完美，而字体风格检测普遍表现极差。模型规模与性能无相关性，且不同难度下的准确率保持一致，表明问题源于训练数据缺失而非模型容量上限。对一个开源模型使用少量合成样本进行LoRA微调，显著提升了性能，使字体大小识别超越最佳闭源系统。然而字体风格识别仍难以改善，暗示关系型视觉推理可能需要突破现有基于块的编码器架构。我们公开了评估框架、数据集及微调方法，以推动视觉语言理解中排版感知的研究。
原文摘要 · Abstract (English)
Vision-Language Models achieve near-perfect accuracy at reading text in images, yet prove largely typography-blind: capable of recognizing what text says, but not how it looks. We systematically investigate this gap by evaluating font family, size, style, and color recognition across 26 fonts, four scripts, and three difficulty levels. Our evaluation of 15 state-of-the-art VLMs reveals a striking perception hierarchy: color recognition is near-perfect, yet font style detection remains universally poor. We further find that model scale fails to predict performance and that accuracy is uniform across difficulty levels, together pointing to a training-data omission rather than a capacity ceiling. LoRA fine-tuning on a small set of synthetic samples substantially improves an open-source model, narrowing the gap to the best closed-source system and surpassing it on font size recognition. Font style alone remains resistant to fine-tuning, suggesting that relational visual reasoning may require architectural innovation beyond current patch-based encoders. We release our evaluation framework, data, and fine-tuning recipe to support progress in closing the typographic gap in vision-language understanding.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。