arXiv:2506.10128cs.CVcs.LG2025-06NeurIPS被引 26

让视觉语言模型学会识别图像描述中的细微错误,提升真实感知能力。

ViCrit: A Verifiable Reinforcement Learning Proxy Task for Visual Perception in VLMs

  • 用人工注入的细微错位描述训练模型定位错误片段。
  • 在多个视觉任务上显著提升表现,且泛化到抽象图像与视觉数学。
  • 适合想提升模型真实理解力的研究者和开发者。

强化学习在大语言模型微调中表现出色,尤其适用于数学推理或代码生成等可验证任务。然而,将此成功扩展至视觉语言模型(VLMs)的视觉感知仍受制于缺乏兼具挑战性与明确可验证性的视觉任务。为此,我们提出ViCrit(视觉描述幻觉批评者),一种用于训练VLMs的强化学习代理任务:在200词的人类撰写图像描述中,注入一个微小的合成视觉错误——仅改变物体、属性、数量或空间关系的几个词——要求模型在给定图像和修改后描述的情况下,准确找出被污染的片段。该设计保持了完整的感知难度,同时提供二值、精确匹配的奖励信号,易于计算且无歧义。使用ViCrit任务训练的模型在多种视觉语言基准测试中均取得显著提升,且性能迁移至自然图像之外的抽象图像推理与视觉数学任务,展现出学习感知而非仅记忆视觉对象的潜力。为便于评估,我们进一步构建了类别平衡的诊断基准ViCrit-Bench,系统性地探测不同图像领域与错误类型下的感知偏差。结果表明,精细的幻觉批评是一种有效且可泛化的视觉感知增强目标。

原文摘要 · Abstract (English)

Reinforcement learning (RL) has shown great effectiveness for fine-tuning large language models (LLMs) using tasks that are challenging yet easily verifiable, such as math reasoning or code generation. However, extending this success to visual perception in vision-language models (VLMs) has been impeded by the scarcity of vision-centric tasks that are simultaneously challenging and unambiguously verifiable. To this end, we introduce ViCrit (Visual Caption Hallucination Critic), an RL proxy task that trains VLMs to localize a subtle, synthetic visual hallucination injected into paragraphs of human-written image captions. Starting from a 200-word captions, we inject a single, subtle visual description error-altering a few words on objects, attributes, counts, or spatial relations-and task the model to pinpoint the corrupted span given the image and the modified caption. This formulation preserves the full perceptual difficulty while providing a binary, exact-match reward that is easy to compute and unambiguous. Models trained with the ViCrit Task exhibit substantial gains across a variety of VL benchmarks. Crucially, the improvements transfer beyond natural-image training data to abstract image reasoning and visual math, showing promises of learning to perceive rather than barely memorizing seen objects. To facilitate evaluation, we further introduce ViCrit-Bench, a category-balanced diagnostic benchmark that systematically probes perception errors across diverse image domains and error types. Together, our results demonstrate that fine-grained hallucination criticism is an effective and generalizable objective for enhancing visual perception in VLMs.

视觉语言模型强化学习感知提升幻觉检测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。