arXiv:2604.24191cs.CV2026-04

提出深度嵌套推理框架,提升多模态复杂任务的思考效率与准确性。

Omni-o3: Deep Nested Omnimodal Deduction for Deliberative Audio-Visual Reasoning

论文配图:Omni-o3: Deep Nested Omnimodal Deduction for Deliberative Audio-Visual Reasoning
图 1 · 摘自论文原文
  • 通过递归搜索共享中间推理路径,实现多分支协同思考。
  • 在11个基准上表现优异,长链推理任务提升显著。
  • 适合需要深度多模态推理的科研与工业场景。

多模态理解涉及海量且高度冗余的跨模态交互搜索空间,亟需聚焦且审慎的推理机制。现有方法依赖串行生成或并行样本独立推演,导致推理轨迹孤立,难以共享有效中间路径,严重限制探索效率并引发复杂任务中的误差累积。为此,本文提出 Omni-o3 框架,采用深层嵌套推理策略,将推理建模为动态递归搜索过程,天然支持各分支间推理前缀共享。该框架迭代执行四种基础认知动作:扩展、选择、模拟与回溯。为支撑此框架,设计两阶段训练方案:(1) 在 101K 条高质量长链轨迹(由 350 万多样本蒸馏而来)上进行冷启动监督微调,学习必要的递归搜索模式;(2) 在 18K 个复杂多轮样本上,通过嵌套组推演驱动的探索性强化学习,并受新型多步奖励模型引导,激发深层嵌套推理。大量实验表明,Omni-o3 在 11 个基准上表现卓越,解锁了全面的音视频、以视觉为主及以音频为主等推理能力。

原文摘要 · Abstract (English)

Omnimodal understanding entails a massive, highly redundant search space of cross-modal interactions, demanding focused and deliberative reasoning. Current reasoning paradigms rely on either sequential step-by-step generation or parallel sample-by-sample rollouts, leading to isolated reasoning trajectories. This inability to share promising intermediate paths severely limits exploration efficiency and causes compounding errors in complex audio-visual tasks. To break this bottleneck, we introduce Omni-o3, a novel framework driven by a deep nested deduction policy. By formulating reasoning as a dynamic recursive search, Omni-o3 inherently shares reasoning prefixes across branches, enabling the iterative execution of four atomic cognitive actions: expansion, selection, simulation, and backpropagation. To empower this framework, we propose a robust two-stage training paradigm: (1) cold-start supervised fine-tuning on 101K high-quality, long-chain trajectories distilled from 3.5M diverse omnimodal samples, enabling necessary recursive search patterns; and (2) nested group rollout-driven exploratory reinforcement learning on 18K complex multi-turn samples, explicitly guided by a novel multi-step reward model to stimulate deep nested reasoning. Extensive experiments demonstrate that Omni-o3 achieves competitive performance across 11 benchmarks, unlocking advanced capabilities in comprehensive audio-visual, visual-centric, and audio-centric reasoning tasks.

多模态推理递归搜索强化学习音视频理解

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。