让视觉推理更高效精准，兼顾细节感知与动态思考过程。
Interleaved Latent Visual Reasoning with Selective Perceptual Modeling
- 用动态潜空间信号交替驱动文本生成与视觉推理
- 在多个基准上超越现有方法，提升显著
- 适合需要精细视觉理解的多模态推理任务
交织式推理范式虽能增强多模态大模型的视觉反馈能力，但因重编码高密度图像导致计算成本过高。潜在视觉推理是可行替代方案，却存在两类局限：或因单步非交织结构无法捕捉中间状态演化，或因过度压缩特征而损失精确感知建模。本文提出交织式潜空间视觉推理（ILVR），统一动态状态演化与精确感知建模。ILVR将文本生成与潜空间视觉表示交替进行，后者作为后续推理的特定、动态提示。具体地，采用自监督策略，通过动量教师模型从真实中间图像中选择性蒸馏相关特征，形成稀疏监督目标。该自适应选择机制引导模型自主生成上下文感知的视觉信号。在多模态推理基准上的大量实验表明，ILVR优于现有方法，有效弥合细粒度感知与序列化多模态推理之间的差距。代码已开源：https://github.com/XD111ds/ILVR。
原文摘要 · Abstract (English)
Interleaved reasoning paradigms enhance Multimodal Large Language Models (MLLMs) with visual feedback but are hindered by the prohibitive computational cost of re-encoding pixel-dense images. A promising alternative, latent visual reasoning, circumvents this bottleneck yet faces limitations: methods either fail to capture intermediate state evolution due to single-step, non-interleaved structures, or sacrifice precise perceptual modeling by over-compressing features. We introduce Interleaved Latent Visual Reasoning (ILVR), a framework that unifies dynamic state evolution with precise perceptual modeling. ILVR interleaves textual generation with latent visual representations that act as specific, evolving cues for subsequent reasoning. Specifically, we employ a self-supervision strategy where a momentum teacher model selectively distills relevant features from ground-truth intermediate images into sparse supervision targets. This adaptive selection mechanism guides the model to autonomously generate context-aware visual signals. Extensive experiments on multimodal reasoning benchmarks demonstrate that ILVR outperforms existing approaches, effectively bridging the gap between fine-grained perception and sequential multimodal reasoning. The code is available at https://github.com/XD111ds/ILVR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。