让视觉推理像作画一样并行生成，提升模型理解图像的能力。
DeepLatent: Think with Images via Parallel Latent Visual Reasoning
- 用可学习的2D令牌并行生成视觉状态，直接锚定在原始图像特征上
- 通过连续空间强化学习优化隐变量，显著提升表征质量
- 适合需要高效精准视觉推理的研究者和开发者
新兴的‘用图像思考’范式将视觉状态嵌入中间推理步骤，为视觉语言模型开辟新方向。现有方法分两条路径：工具辅助方法虽能执行显式视觉操作，但延迟高且操作类型受限；隐式推理方法虽可自回归生成隐状态，但性能落后，且隐变量无法有效捕捉视觉信息。本文提出DeepLatent，一种并行的隐式视觉推理框架。首先，引入LatentFormer，利用可学习的2D令牌并行生成上下文相关的隐状态，将每次视觉更新直接锚定在原始图像特征中。其次，设计一种连续空间强化学习算法，直接在嵌入空间优化隐变量调节参数，显著提升隐表示质量。框架通过知识蒸馏训练后，再经此连续空间强化学习优化。此外，我们构建了大规模数据集DeepLatent-180K，专用于隐式视觉推理。多基准测试表明，DeepLatent达到当前最优性能。
原文摘要 · Abstract (English)
The emerging paradigm of "thinking with images" embeds visual states into intermediate reasoning steps, defining a new frontier for Vision-Language Models. Existing approaches diverge along two lines. Tool-assisted methods apply explicit visual operations but suffer from high latency and restricted manipulation types. Latent reasoning methods autoregressively produce implicit visual states, but underperform tool-assisted methods, and their latent tokens fail to capture effective visual information. In this work, we propose DeepLatent, a parallel framework for latent visual reasoning. First, we introduce LatentFormer. It uses learnable 2D tokens to generate context-conditioned latent states in parallel, anchoring every visual update directly in the original image features. Second, we design a continuous-space reinforcement learning algorithm. It optimizes latent modulation parameters directly in the embedding space, significantly improving latent representation quality. The framework is trained via knowledge distillation followed by this continuous-space RL algorithm. Furthermore, we contribute DeepLatent-180K, a large-scale dataset tailored for latent visual reasoning. Extensive evaluations across multiple benchmarks demonstrate that DeepLatent achieves state-of-the-art performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。