解决多模态模型在文档模糊时的误识别问题,提升判断不确定性的能力。
Seeing is Believing? Mitigating OCR Hallucinations in Multimodal Large Language Models
- 设计新奖励机制,让模型自知视觉不确定性并拒绝回答模糊信息。
- 在退化文档上实现22%的幻觉消除准确率提升,标准任务性能不变。
- 构建首个专门评估OCR幻觉的基准数据集,覆盖身份证与发票场景。
多模态大模型虽通过融合文本与视觉信息提升了文档理解能力,但在真实场景中面对视觉退化时仍存在感知不足的问题。当前模型常过度依赖语言先验或产生视觉-文本推理错位,难以识别不确定性,导致生成幻觉内容。为此,我们提出KIE-HVQA,首个专注于评估退化文档中OCR幻觉的基准数据集,包含身份卡和发票样本,并模拟真实世界退化以测试OCR可靠性。该设置可评估模型在退化输入下区分可信视觉信息并合理作答的能力,揭示幻觉风险。为实现视觉忠实推理,我们引入基于GRPO的框架,结合视觉不确定性自知与主动拒绝回答机制,在监督微调与强化学习中增强模型鲁棒性。在Qwen2.5-VL上的实验表明,7B参数模型相较GPT-4o在KIE-HVQA上实现22%绝对幻觉消除准确率提升,且标准任务性能无显著下降,验证了方法的有效性与鲁棒性。
原文摘要 · Abstract (English)
Recent advancements in multimodal large language models have enhanced document understanding by integrating textual and visual information. However, existing models exhibit incompleteness within their paradigm in real-world scenarios, particularly under visual degradation. In such conditions, the current response paradigm often fails to adequately perceive visual degradation and ambiguity, leading to overreliance on linguistic priors or misaligned visual-textual reasoning. This difficulty in recognizing uncertainty frequently results in the generation of hallucinatory content, especially when a precise answer is not feasible. To better demonstrate and analyze this phenomenon and problem, we propose KIE-HVQA, the first benchmark dedicated to evaluating OCR hallucination in degraded document understanding. This dataset includes test samples spanning identity cards and invoices, with simulated real-world degradations for OCR reliability. This setup allows for evaluating models' capacity, under degraded input, to distinguish reliable visual information and answer accordingly, thereby highlighting the challenge of avoiding hallucination on uncertain data. To achieve vision-faithful reasoning and thereby avoid the aforementioned issues, we further introduce a GRPO-based framework featuring a novel reward mechanism. By incorporating a self-awareness of visual uncertainty and an analysis method that initiates refusal to answer to increase task difficulty within our supervised fine-tuning and reinforcement learning framework, we successfully mitigated hallucinations in ambiguous regions. Experiments on Qwen2.5-VL demonstrate that our 7B-parameter model achieves a 22\% absolute improvement in hallucination-free accuracy over GPT-4o on KIE-HVQA and there is no significant performance drop in standard tasks, highlighting both effectiveness and robustness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。