让多模态大模型自动判断何时思考，提升效率与性能。
R-4B: Incentivizing General-Purpose Auto-Thinking Capability in MLLMs via Bi-Mode Annealing and Reinforce Learning
- 用双模式渐进训练让模型学会自主选择思考或直接回答。
- 在25个基准上表现超越Qwen2.5-VL-7B，接近16B大模型效果。
- 适合需要高效推理的复杂视觉语言任务场景。
具备逐步思考能力的多模态大语言模型在复杂推理任务中表现优异，但对简单问题仍进行冗余思考。为此，我们提出R-4B，一种可自适应决定是否启用思考的多模态大模型。核心思路是通过双模式渐进训练，赋予模型思考与非思考双重能力，并采用双模式策略优化（BPO）提升其判断是否激活思考过程的准确性。首先，在涵盖多种主题的精心构建数据集上训练模型，包含思考与非思考模式样本；随后在改进的GRPO框架下进行第二阶段训练，强制策略模型对每个输入同时生成两种模式的响应。实验结果表明，R-4B在25个挑战性基准上达到当前最优性能，多数任务优于Qwen2.5-VL-7B，且在推理密集型任务上表现接近16B级模型Kimi-VL-A3B-Thinking-2506，同时计算开销更低。
原文摘要 · Abstract (English)
Multimodal Large Language Models (MLLMs) equipped with step-by-step thinking capabilities have demonstrated remarkable performance on complex reasoning problems. However, this thinking process is redundant for simple problems solvable without complex reasoning. To address this inefficiency, we propose R-4B, an auto-thinking MLLM, which can adaptively decide when to think based on problem complexity. The central idea of R-4B is to empower the model with both thinking and non-thinking capabilities using bi-mode annealing, and apply Bi-mode Policy Optimization (BPO) to improve the model's accuracy in determining whether to activate the thinking process. Specifically, we first train the model on a carefully curated dataset spanning various topics, which contains samples from both thinking and non-thinking modes. Then it undergoes a second phase of training under an improved GRPO framework, where the policy model is forced to generate responses from both modes for each input query. Experimental results show that R-4B achieves state-of-the-art performance across 25 challenging benchmarks. It outperforms Qwen2.5-VL-7B in most tasks and achieves performance comparable to larger models such as Kimi-VL-A3B-Thinking-2506 (16B) on reasoning-intensive benchmarks with lower computational cost.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。