零标注自进化多模态推理框架，让模型自己出题自己解题
V-Zero: Self-Improving Multimodal Reasoning with Zero Annotation
- 通过提问者与求解者双角色协同，利用无标注图像实现自我提升
- 在Qwen2.5-VL-7B上视觉数学推理提升1.7，通用视觉任务提升2.6
- 无需人工标注，适合追求低成本自优化的多模态研究者
多模态学习的最新进展显著提升了视觉语言模型（VLMs）的推理能力。然而，当前先进方法严重依赖大规模人工标注数据集，获取成本高且耗时。为克服这一局限，我们提出V-Zero——一种通用的后训练框架，仅使用无标注图像即可实现自进化。V-Zero通过构建双角色协同进化机制：提问者学习生成高质量、具有挑战性的题目，采用双轨推理奖励对比直觉猜测与理性推断结果；求解者则基于自身采样响应的多数投票生成伪标签进行优化。两者通过组相对策略优化（GRPO）迭代训练，形成相互增强的闭环。令人瞩目的是，完全无需人工标注，V-Zero在Qwen2.5-VL-7B-Instruct上实现了视觉数学推理+1.7、通用视觉任务+2.6的持续性能提升，展示了多模态系统自进化潜力。代码已开源。
原文摘要 · Abstract (English)
Recent advances in multimodal learning have significantly enhanced the reasoning capabilities of vision-language models (VLMs). However, state-of-the-art approaches rely heavily on large-scale human-annotated datasets, which are costly and time-consuming to acquire. To overcome this limitation, we introduce V-Zero, a general post-training framework that facilitates self-improvement using exclusively unlabeled images. V-Zero establishes a co-evolutionary loop by instantiating two distinct roles: a Questioner and a Solver. The Questioner learns to synthesize high-quality, challenging questions by leveraging a dual-track reasoning reward that contrasts intuitive guesses with reasoned results. The Solver is optimized using pseudo-labels derived from majority voting over its own sampled responses. Both roles are trained iteratively via Group Relative Policy Optimization (GRPO), driving a cycle of mutual enhancement. Remarkably, without a single human annotation, V-Zero achieves consistent performance gains on Qwen2.5-VL-7B-Instruct, improving visual mathematical reasoning by +1.7 and general vision-centric by +2.6, demonstrating the potential of self-improvement in multimodal systems. Code is available at https://github.com/SatonoDia/V-Zero
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。