arXiv:2606.18974cs.CV2026-06被引 5

用自蒸馏让模型不生成图像也能学会视觉推理,速度快14倍且更准。

Visual-OPSD: Cross-Modal On-Policy Self-Distillation for Efficient Unified Multimodal Reasoning

论文配图:Visual-OPSD: Cross-Modal On-Policy Self-Distillation for Efficient Unified Multimodal Reasoning
图 1 · 摘自论文原文
  • 让学生模型通过模仿教师的生成过程来学习视觉推理,无需实际生成图像。
  • 在9个任务上提升3.4个百分点,推理速度加快14.3倍,仅需10秒/样本。
  • 适合追求高效多模态推理的开发者,尤其关注速度与精度平衡的场景。

统一多模态模型(UMMs)通过交替生成‘视觉思维’(VTs)与文本推理来提升空间理解能力,但多步扩散过程带来约一个数量级的推理开销。我们发现这种开销带来的直接收益有限:在ThinkMorph上,删除或噪声化VTs对九项基准测试的准确率影响极小。一旦生成,注意力几乎完全集中在VT像素上,与其内容无关。然而KL诊断显示,依赖特权VT轨迹会显著改变模型的输出分布,表明生成路径本身蕴含了超越渲染图像的有用推理信息。为此,我们提出视觉在线自蒸馏(Visual-OPSD)。教师与学生共享相同权重,但教师可见特权VTs,学生仅见问题。在学生采样轨迹上进行基于令牌级别的JSD蒸馏,将教师的推理能力传递给纯文本学生。在九个基准上,Visual-OPSD以14.3倍的速度提升(10.0秒 vs. 142.8秒/样本)获得+3.40个百分点的性能增益,并在VSP上比同规模VLM高出+63.83个百分点。高斯噪声对照实验(+0.40个百分点)与58.4%的KL差距缩小证实,性能提升源于生成路径中的语义内容。

原文摘要 · Abstract (English)

Unified multimodal models (UMMs) interleave generated ''visual thoughts'' (VTs) with text reasoning to improve spatial tasks. This incurs roughly an order-of-magnitude inference cost from multi-step diffusion. We find this cost yields limited direct benefit. On ThinkMorph, removing or noising VTs barely changes accuracy across nine benchmarks. Once rendered, attention concentrates on the VT regardless of content. Yet a KL diagnostic shows that conditioning on a privileged VT trace shifts the model's completion distribution. This suggests the generation pathway encodes useful reasoning beyond the rendered pixels. Motivated by this gap, we propose Visual On-Policy Self-Distillation(Visual-OPSD). Teacher and student share identical weights but differ in context: the teacher sees privileged VTs while the student sees only the question. Token-level JSD distillation on on-policy student trajectories transfers the teacher's reasoning to a text-only student. Across nine benchmarks, Visual-OPSD improves over its generative teacher by $+3.40$pp with $14.3\times$ speedup (10.0s vs. 142.8s per sample) and outperforms same-scale VLMs by $+63.83$pp on VSP. A Gaussian-noise control ($+0.40$pp vs. $+10.28$pp for real VTs) and $58.4\%$ closure of the KL gap confirm that gains come from the semantic content of the generation pathway.

多模态推理自蒸馏效率优化视觉思维

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。