VLMs常篡改模糊文本，而非忠实转录，影响文档理解可靠性。
Do VLMs Read or Rewrite? On Transcription Faithfulness in Vision-Language Models
- 构建多语言扰动数据集FaithC4，模拟真实文本错误
- 通用VLM在英文上误写率最高达10%，长词基本不改
- 模型重写行为由语义编码是否接近决定，适合文档质检场景
视觉语言模型（VLM）正逐步替代传统OCR用于文档理解。本文揭示其并非始终忠实转录：当文本模糊时，模型倾向于将其改写为更合理的形式——这一行为无法被干净文本的OCR基准检测到。为此，我们提出FaithC4，一个包含1,455页单页文档的多语言扰动基准，涵盖英语、中文、韩语，包含三种扰动类型：打乱、随机替换和视觉相似替换。我们评估了15种系统，包括通用VLM、专用于OCR的VLM及传统OCR流程。三类系统在扰动下的词错误率（WER）退化差异显著：通用VLM最高退化4.5点，专用VLM退化0.2-2点，传统OCR低于0.6点（英文）。对Qwen3-VL-4B逐层探查发现，仅当扰动词的最终层前馈网络（FFN）表示仍接近原始编码时才会发生重写；若表示足够发散，则模型忠实转录。词长影响重写率：4-6字符短词重写率达10%，8字符以上则重写率为0%。
原文摘要 · Abstract (English)
Vision Language Models (VLMs) are increasingly used in place of traditional OCR pipelines for document understanding. In this paper, we show they do not always act as faithful transcribers: when text is imperfect, they often tend to rewrite it into a more plausible form - a behavior that clean-text OCR benchmarks cannot detect. We introduce FaithC4, a multilingual perturbation benchmark of 1,455 single-page documents (English, Chinese, Korean) with three perturbation families: scramble, random substitution, and visually similar substitution. We use the benchmark to evaluate 15 systems spanning general-purpose VLMs, OCR-specialized VLMs, and traditional OCR pipelines. These three categories differ in WER degradation under perturbation: general-purpose VLMs degrade by up to 4.5 points, OCR-specialized VLMs by 0.2-2 points, and traditional OCR by less than 0.6 points on English. Probing Qwen3-VL-4B layer-by-layer, we identify a consistent pattern: rewriting fires only when a perturbed word's final layer FFN representation stays close to the original encoding; when the representation diverges sufficiently, the model transcribes faithfully. Word length affects rewriting rate: short words (4-6 characters) are rewritten up to 10% of the time, with a sharp cutoff at 8 characters above which rewriting drops to 0%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。