arXiv:2504.07934cs.CV2025-04NeurIPS被引 149

用少样本实现视觉推理新SOTA,靠MCTS选难但可解的题目。

SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvement

论文配图:SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvement
图 1 · 摘自论文原文
  • 用MCTS衡量样本难度,筛选出最能促进推理提升的样本。
  • 仅用11k/7.5k样本训练,7B和72B模型均达新SOTA。
  • 适合追求高效训练与强推理能力的研究者和开发者。

我们提出ThinkLite-VL,一系列在极少训练样本下达到状态领先(SoTA)性能的视觉推理模型,仅通过强化学习微调(RFT)自我改进,无需知识蒸馏。核心洞察是:样本难度显著影响RFT效果——适当挑战性的例题可驱动大幅推理提升,即使在低数据条件下亦然。为量化难度,我们复用蒙特卡洛树搜索(MCTS)测量视觉语言模型(VLM)解决每个实例所需的推理迭代次数,以此筛选出诱导深度推理但仍可解的高质量样本。从7万开源样本中筛选出11,000个用于Qwen2.5-VL-7B-Instruct的RFT,7,500个用于Qwen2.5-VL-72B-Instruct。结果模型在八大视觉推理基准上显著超越基线。ThinkLite-VL-7B相较基线平均提升7%,在MathVista上达75.1分,超越所有7B级模型及更大模型如GPT-4o、O1和Qwen2.5-VL-72B;ThinkLite-VL-72B在MathVista达79.7分,平均提升4.42分,进一步刷新开源模型纪录。该方法为多模态推理的数据高效自提升提供了可扩展有效路径。

原文摘要 · Abstract (English)

We introduce ThinkLite-VL, a family of visual reasoning models that achieve state-of-the-art (SoTA) performance using an order of magnitude fewer training samples, relying purely on reinforcement fine-tuning (RFT) self-improvement without any knowledge distillation. Our central insight is that sample difficulty critically influences RFT effectiveness: appropriately challenging examples can drive substantial reasoning improvements, even in low-data regimes. However, quantifying sample difficulty in a reliable and scalable manner remains non-trivial. To address this, we repurpose Monte Carlo Tree Search (MCTS) to measure sample difficulty via the number of reasoning iterations a vision-language model (VLM) requires to solve each instance. This MCTS-based selection procedure identifies samples that induce deeper reasoning while remaining solvable, allowing us to filter a high-quality subset from 70k open-source examples spanning math, natural image understanding, and chart comprehension. Using this approach, we select just 11k challenging samples for RFT on Qwen2.5-VL-7B-Instruct and 7.5k samples for Qwen2.5-VL-72B-Instruct. The resulting models, ThinkLite-VL-7B and ThinkLite-VL-72B, significantly outperform their respective base models across eight visual reasoning benchmarks. In particular, ThinkLite-VL-7B improves the average performance of Qwen2.5-VL-7B-Instruct by 7\% and surpasses all existing 7B-level models, as well as much larger models such as GPT-4o, O1 and Qwen2.5-VL-72B, achieving a new SoTA score of 75.1 on MathVista. ThinkLite-VL-72B further advances the SoTA frontier, achieving an accuracy of 79.7 on MathVista and an average benchmark improvement of 4.42 over the open-source SOTA. These results demonstrate that MCTS-guided difficulty filtering provides a scalable and effective path toward data-efficient self-improvement in multimodal reasoning.

视觉推理强化学习少样本MCTS

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。