arXiv:2512.02456cs.CVcs.CL2025-12中稿 · The Winter Confere…被引 2

让视觉语言模型先看后想,自动生成推理链条提升多模态推理能力。

See, Think, Learn: A Self-Taught Multimodal Reasoner

  • 引入结构化推理模板,强制模型先提取视觉特征再推理。
  • 自训练循环中联合优化感知与推理,性能超越纯答案或简单推理训练。
  • 加入错误解释作为负样本,增强模型辨别正确与误导性答案的能力。

视觉语言模型(VLMs)在融合视觉感知与语言理解方面取得了显著进展。然而,有效的多模态推理需要准确的感知和强大的推理能力,任一环节薄弱都会限制模型表现。以往提升推理的方法通常依赖高质量的思维链(CoT)数据,这些数据通过人工标注、昂贵专有模型或忽略感知的自训练方法获得。为此,我们提出一种简单而高效的自训练框架——看-想-学(See-Think-Learn, STL)。核心在于引入结构化推理模板,引导模型先‘看’再‘想’,首先以文本形式提取视觉属性,再利用这些属性指导推理。该框架通过模型自生成并学习结构化推理过程,在自训练循环中联合提升感知与推理能力。此外,我们还引入负向推理(即解释为何某些选项错误),增强模型区分正确与误导性回答的能力,实现更具判别力和鲁棒的学习。跨多个领域的实验表明,STL持续优于仅基于答案或自生成推理训练的基线模型,定性分析也证实其推理链条质量高。STL为提升VLM的多模态推理能力提供了一种低成本有效方案。

原文摘要 · Abstract (English)

Vision-Language Models (VLMs) have achieved remarkable progress in integrating visual perception with language understanding. However, effective multimodal reasoning requires both accurate perception and robust reasoning, and weakness in either limits the performance of VLMs. Prior efforts to enhance reasoning often depend on high-quality chain-of-thought (CoT) data, obtained via labor-intensive human annotations, costly proprietary models, or self-training methods that overlook perception. To address these limitations, we propose a simple yet effective self-training framework called See-Think-Learn (STL). At its core, STL introduces a structured reasoning template that encourages the model to see before thinking, first extracting visual attributes in textual form, then using them to guide reasoning. The framework jointly improves perception and reasoning by having the model generate and learn from its own structured rationales in a self-training loop. Furthermore, we augment the training data with negative rationales, i.e. explanations that justify why certain answer choices are incorrect, to enhance the model's ability to distinguish between correct and misleading responses. This fosters more discriminative and robust learning. Experiments across diverse domains show that STL consistently outperforms baselines trained directly only on answers or self-generated reasoning, while qualitative analysis confirms the high quality of its rationales. STL thus provides a cost-effective solution to enhance multimodal reasoning ability of VLMs.

多模态推理自训练视觉语言模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。