让视觉语言模型自动纠错，用少量数据就能越改越好。
Sherlock: Self-Correcting Reasoning in Vision-Language Models
- 通过轨迹级自纠正机制，让模型自己发现并修正推理错误。
- 仅用2万条标注数据，平均准确率达64.1，自纠正后提升至65.4。
- 无需外部监督，适合希望降低标注成本的研究者使用。
推理型视觉语言模型（VLMs）在复杂多模态任务中表现优异，但仍面临易受推理错误影响、依赖大量标注数据或精确验证器、泛化能力弱等问题。为解决这些挑战，本文探索自纠正策略以增强推理型VLMs。我们首先深入分析了现有模型的自纠正能力并识别关键差距。基于此，提出Sherlock框架，包含轨迹级自纠正目标、基于视觉扰动的偏好数据构建方法，以及动态β偏好调优机制。模型仅需20,000条随机采样的标注数据即可获得自纠正能力，并持续自我优化而无需外部监督。基于Llama3.2-Vision-11B，Sherlock在八个基准测试上达到64.1的直接生成平均准确率，自纠正后提升至65.4，优于LLaVA-CoT（63.2）、Mulberry（63.9）和LlamaV-o1（63.4），且所用标注数据不足前者的20%。
原文摘要 · Abstract (English)
Reasoning Vision-Language Models (VLMs) have shown promising performance on complex multimodal tasks. However, they still face significant challenges: they are highly sensitive to reasoning errors, require large volumes of annotated data or accurate verifiers, and struggle to generalize beyond specific domains. To address these limitations, we explore self-correction as a strategy to enhance reasoning VLMs. We first conduct an in-depth analysis of reasoning VLMs' self-correction abilities and identify key gaps. Based on our findings, we introduce Sherlock, a self-correction and self-improvement training framework. Sherlock introduces a trajectory-level self-correction objective, a preference data construction method based on visual perturbation, and a dynamic $β$ for preference tuning. Once the model acquires self-correction capabilities using only 20k randomly sampled annotated data, it continues to self-improve without external supervision. Built on the Llama3.2-Vision-11B model, Sherlock achieves remarkable results across eight benchmarks, reaching an average accuracy of 64.1 with direct generation and 65.4 after self-correction. It outperforms LLaVA-CoT (63.2), Mulberry (63.9), and LlamaV-o1 (63.4) while using less than 20% of the annotated data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。