让大模型在图像隐空间中进行类人抽象推理,突破传统图文结合的局限。
Monet: Reasoning in Latent Visual Space Beyond Images and Language
- 通过生成连续嵌入实现视觉隐空间中的中间思考步骤
- 在125K真实场景推理数据上训练,提升跨域泛化能力
- 提出新强化学习方法VLPO,专门优化隐空间推理
‘用图像思考’作为一种新兴范式,通过在推理过程中引入视觉证据,拓展了仅依赖文本的链式思维。然而,现有方法受限于外部工具,难以实现类人抽象视觉思维。本文提出Monet训练框架,使多模态大模型可在潜空间中直接进行视觉推理,通过生成连续嵌入作为中间视觉思想。我们识别出两大挑战:潜视觉对齐计算开销大、潜嵌入监督不足,并设计三阶段基于蒸馏的有监督微调(SFT)流程解决。进一步发现GRPO在潜空间推理中效果有限,因此提出VLPO(视觉潜空间策略优化),将潜嵌入显式纳入策略梯度更新。为支持SFT,构建了包含12.5万条真实世界、图表、OCR与几何推理的高质量文本-图像交错思维链数据集Monet-SFT-125K。Monet-7B模型在真实感知与推理基准上表现持续提升,在复杂抽象视觉推理任务中展现强泛化能力。我们还分析各训练组件作用,并讨论早期失败尝试,为未来视觉潜空间推理研究提供洞见。模型、数据与代码已开源。
原文摘要 · Abstract (English)
"Thinking with images" has emerged as an effective paradigm for advancing visual reasoning, extending beyond text-only chains of thought by injecting visual evidence into intermediate reasoning steps. However, existing methods fall short of human-like abstract visual thinking, as their flexibility is fundamentally limited by external tools. In this work, we introduce Monet, a training framework that enables multimodal large language models (MLLMs) to reason directly within the latent visual space by generating continuous embeddings that function as intermediate visual thoughts. We identify two core challenges in training MLLMs for latent visual reasoning: high computational cost in latent-vision alignment and insufficient supervision over latent embeddings, and address them with a three-stage distillation-based supervised fine-tuning (SFT) pipeline. We further reveal a limitation of applying GRPO to latent reasoning: it primarily enhances text-based reasoning rather than latent reasoning. To overcome this, we propose VLPO (Visual-latent Policy Optimization), a reinforcement learning method that explicitly incorporates latent embeddings into policy gradient updates. To support SFT, we construct Monet-SFT-125K, a high-quality text-image interleaved CoT dataset containing 125K real-world, chart, OCR, and geometry CoTs. Our model, Monet-7B, shows consistent gains across real-world perception and reasoning benchmarks and exhibits strong out-of-distribution generalization on challenging abstract visual reasoning tasks. We also empirically analyze the role of each training component and discuss our early unsuccessful attempts, providing insights for future developments in visual latent reasoning. Our model, data, and code are available at https://github.com/NOVAglow646/Monet.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。