arXiv:2504.15362cs.CVcs.CL2025-04被引 17

让视觉模型像人一样慢思考,提升复杂视觉推理能力

LongPerceptualThoughts: Distilling System-2 Reasoning for System-1 Perception

  • 构建3万条长思维链的合成数据集,模拟深度视觉推理过程
  • 在5个视觉基准上平均提升3.4分,V$^*$ Bench最高提升11.8分
  • 不仅改善视觉任务,还让文本推理能力提升2分,适合多模态研究者

近期通过测试时扩展的推理模型表明,长链式思维可在数学和编程等难题中显著提升性能。然而,这类长思维对感知类任务(如视觉理解)的价值尚未被充分探索,因浅层、快速的系统-1推理通常已足够。本文提出LongPerceptualThoughts,一个包含3万条长思维链的合成数据集,用于感知任务。由于现成模型缺乏深度思考行为,且感知任务难以构建可靠验证机制,我们设计三阶段数据生成框架:首先从密集图像描述生成可验证的多选题;然后利用视觉语言模型提取简单思维链;最后通过前沿推理模型扩展为详尽的长思维链。在强指令微调的7B模型上进行控制实验,结果表明该方法优于现有视觉推理数据生成方法。基于该数据集训练的模型,在5个以视觉为核心的基准测试中平均提升3.4分,其中V$^*$ Bench提升11.8分。值得注意的是,尽管专为视觉任务优化,该模型在文本推理基准MMLU-Pro上也实现了+2分的提升。

原文摘要 · Abstract (English)

Recent reasoning models through test-time scaling have demonstrated that long chain-of-thoughts can unlock substantial performance boosts in hard reasoning tasks such as math and code. However, the benefit of such long thoughts for system-2 reasoning is relatively less explored in other domains such as perceptual tasks where shallower, system-1 reasoning seems sufficient. In this paper, we introduce LongPerceptualThoughts, a new synthetic dataset with 30K long-thought traces for perceptual tasks. The key challenges in synthesizing elaborate reasoning thoughts for perceptual tasks are that off-the-shelf models are not yet equipped with such thinking behavior and that it is not straightforward to build a reliable process verifier for perceptual tasks. Thus, we propose a novel three-stage data synthesis framework that first synthesizes verifiable multiple-choice questions from dense image descriptions, then extracts simple CoTs from VLMs for those verifiable problems, and finally expands those simple thoughts to elaborate long thoughts via frontier reasoning models. In controlled experiments with a strong instruction-tuned 7B model, we demonstrate notable improvements over existing visual reasoning data-generation methods. Our model, trained on the generated dataset, achieves an average +3.4 points improvement over 5 vision-centric benchmarks, including +11.8 points on V$^*$ Bench. Notably, despite being tuned for vision tasks, it also improves performance on the text reasoning benchmark, MMLU-Pro, by +2 points.

视觉推理长思维链数据合成多模态

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。