DeepSeek-OCR性能依赖语言先验,视觉能力有限。
Visual Merit or Linguistic Crutch? A Close Look at DeepSeek-OCR
- 通过语义干扰测试分离视觉与语言能力
- 无语言支持时准确率从90%降至20%
- 适合关注多模态模型可信度的研究者
DeepSeek-OCR采用二维光学映射实现高比率视觉-文本压缩,声称可解码超过输入视觉标记十倍的文本标记。尽管这看似缓解了大模型长上下文瓶颈,但本文探究核心问题:性能由视觉能力还是语言先验驱动?通过句级与词级语义破坏实验,我们分离模型内在OCR能力与语言先验。结果表明,缺乏语言支持时,模型性能从约90%骤降至20%。对比13种基线模型发现,传统流水线OCR方法对语义扰动更具鲁棒性,而端到端方法更易受先验影响。此外,视觉标记越少,对先验依赖越强,幻觉风险越高。上下文压力测试显示,模型在约10,000文本标记处出现整体崩溃,提示当前光学压缩技术可能反而加剧长上下文瓶颈。本研究实证界定了DeepSeek-OCR的能力边界,并为视觉-文本压缩范式优化提供关键洞见。所有数据、结果与脚本已公开于https://github.com/dududuck00/DeepSeekOCR。
原文摘要 · Abstract (English)
DeepSeek-OCR utilizes an optical 2D mapping approach to achieve high-ratio vision-text compression, claiming to decode text tokens exceeding ten times the input visual tokens. While this suggests a promising solution for the LLM long-context bottleneck, we investigate a critical question: "Visual merit or linguistic crutch - which drives DeepSeek-OCR's performance?" By employing sentence-level and word-level semantic corruption, we isolate the model's intrinsic OCR capabilities from its language priors. Results demonstrate that without linguistic support, DeepSeek-OCR's performance plummets from approximately 90% to 20%. Comparative benchmarking against 13 baseline models reveals that traditional pipeline OCR methods exhibit significantly higher robustness to such semantic perturbations than end-to-end methods. Furthermore, we find that lower visual token counts correlate with increased reliance on priors, exacerbating hallucination risks. Context stress testing also reveals a total model collapse around 10,000 text tokens, suggesting that current optical compression techniques may paradoxically aggravate the long-context bottleneck. This study empirically defines DeepSeek-OCR's capability boundaries and offers essential insights for future optimizations of the vision-text compression paradigm. We release all data, results and scripts used in this study at https://github.com/dududuck00/DeepSeekOCR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。