arXiv:2608.19669cs.CVcs.LG2026-08

优化视觉推理的潜在表示,提升多模态任务表现。

Scaffolding Minds: Optimizing Latent Visual Target Representations for Multimodal Reasoning

论文配图:Scaffolding Minds: Optimizing Latent Visual Target Representations for Multimodal Reasoning
图 1 · 摘自论文原文
  • 设计专用编码器生成更适配推理任务的潜在目标
  • 在强化学习中同时学习采样均值与方差,增强探索能力
  • 在多个视觉推理任务上显著超越现有基线

隐式推理通过两阶段训练范式推动多模态推理发展:(1) 在监督微调(SFT)阶段,使用现成视觉编码器将辅助图像编码为潜在标记,以传授视觉思维链;(2) 在强化学习(RL)阶段,利用奖励反馈进一步优化这些潜在标记。本文发现该框架在两个阶段存在关键局限:第一,SFT阶段通常依赖预训练视觉编码器,导致潜在表示次优且与下游推理任务对齐不足;第二,现有RL方法仅通过确定性正则化处理潜在成分,虽限制策略漂移,但无法生成替代潜在轨迹以支持探索。为此,我们提出Scaffolding Minds。该方法学习一个专用的支架编码器,生成优化后的潜在空间目标,并在RL阶段同时学习采样器的均值与方差。实验表明,两项改进互补性强,共同带来显著提升:在FrozenLake空间规划任务上,相比最强基线提升9.5%;在32x32网格地图下,提升扩大至19%;在九个以视觉为中心的推理基准上平均提升5.2%。

原文摘要 · Abstract (English)

Latent reasoning has advanced multimodal reasoning through a two-stage training paradigm: (1) a helper image is encoded into latent tokens to teach visual chain-of-thought during a supervised fine-tuning (SFT) stage, and (2) these latent tokens are further refined with reward feedback during a reinforcement learning (RL) stage. In this paper, we identify two key limitations of this framework, one in each stage. First, the SFT stage typically relies on an off-the-shelf vision encoder to encode the helper image, yielding suboptimal latent representations that may not be well aligned with the downstream reasoning task. Second, existing RL methods treat the latent component only through deterministic regularization, which constrains policy drift but does not create alternative latent trajectories for exploration. To address these limitations, we propose Scaffolding Minds. Our approach learns a dedicated scaffolding encoder that provides an optimized target in latent space, and learns both the mean and variance of the RL sampler. We further show that these two improvements are complementary, together yielding substantial gains over strong baselines. Empirically, our method improves over the strongest latent-reasoning baseline by +9.5% on FrozenLake spatial planning, with the gain widening to +19% at 32x32 grid map, and by +5.2% on average across nine visual-centric reasoning benchmarks.

多模态推理潜在表示强化学习视觉链式思维

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。