arXiv:2509.07969cs.CVcs.AI2025-09被引 111

让AI在视觉搜索中像人一样反复试错,实现深度推理

Mini-o3: Scaling Up Reasoning Patterns and Interaction Turns for Visual Search

  • 构建挑战性视觉搜索数据集,支持多轮探索式推理
  • 训练仅限6步,推理时自然扩展至数十步,准确率随步骤提升
  • 适合需要深度思考的复杂视觉任务研究者

近期大型多模态模型通过基于图像的工具与强化学习解决视觉问题。但现有开源方法推理模式单一,交互轮次有限,难以应对需试错探索的复杂任务。本文提出Mini-o3系统,通过扩大工具交互轮次,实现长达数十步的深度多轮推理,在复杂视觉搜索任务上达到顶尖表现。复现OpenAI o3行为的关键包括:构建包含数千个挑战性视觉搜索问题的视觉探针数据集;设计迭代式数据收集流程,获取包含深度优先搜索、试错和目标保持等多样推理模式的冷启动轨迹;提出超轮次掩码策略,在强化学习中避免对超过最大轮次响应的惩罚,兼顾训练效率与推理可扩展性。尽管训练上限仅为6轮交互,模型在推理时能自然扩展至数十轮,准确率随轮次增加而提升。大量实验表明,Mini-o3生成丰富推理模式与深层思维路径,有效解决复杂视觉搜索问题。

原文摘要 · Abstract (English)

Recent advances in large multimodal models have leveraged image-based tools with reinforcement learning to tackle visual problems. However, existing open-source approaches often exhibit monotonous reasoning patterns and allow only a limited number of interaction turns, making them inadequate for difficult tasks that require trial-and-error exploration. In this work, we address this limitation by scaling up tool-based interactions and introduce Mini-o3, a system that executes deep, multi-turn reasoning -- spanning tens of steps -- and achieves state-of-the-art performance on challenging visual search tasks. Our recipe for reproducing OpenAI o3-style behaviors comprises three key components. First, we construct the Visual Probe Dataset, a collection of thousands of challenging visual search problems designed for exploratory reasoning. Second, we develop an iterative data collection pipeline to obtain cold-start trajectories that exhibit diverse reasoning patterns, including depth-first search, trial-and-error, and goal maintenance. Third, we propose an over-turn masking strategy that prevents penalization of over-turn responses (those that hit the maximum number of turns) during reinforcement learning, thereby balancing training-time efficiency with test-time scalability. Despite training with an upper bound of only six interaction turns, our model generates trajectories that naturally scale to tens of turns at inference time, with accuracy improving as the number of turns increases. Extensive experiments demonstrate that Mini-o3 produces rich reasoning patterns and deep thinking paths, effectively solving challenging visual search problems.

视觉搜索多轮推理强化学习深度思考

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。