arXiv:2604.26934cs.CV2026-04被引 1

用世界模型教视觉语言模型想象动态场景变化,提升空间推理能力。

World2VLM: Distilling World Model Imagination into VLMs for Dynamic Spatial Reasoning

论文配图:World2VLM: Distilling World Model Imagination into VLMs for Dynamic Spatial Reasoning
图 1 · 摘自论文原文
  • 通过生成世界模型合成未来视角,为VLM提供结构化训练信号。
  • 在多个基准上显著超越基线模型,最高提升12.3%准确率。
  • 无需推理时生成,适合需要高效动态推理的应用场景。

视觉语言模型(VLM)在静态视觉理解上表现优异,但在需要模拟自我中心运动下场景演化的动态空间推理任务中仍显不足。现有方法或依赖合成数据扩大空间监督,或在推理时耦合世界模型,前者缺乏运动条件下的状态转移建模,后者计算开销大。本文提出World2VLM,一种将生成式世界模型的空间想象能力蒸馏至VLM的训练框架。给定初始观测和参数化相机轨迹,利用视图一致的世界模型合成几何对齐的未来视角,并生成前向(动作到结果)与逆向(结果到动作)的空间推理结构化监督。在该流水线生成的小规模数据集上进行两阶段微调后,我们在SAT-Real、SAT-Synthesized、VSI-Bench和MindCube等多个空间推理基准上评估模型。World2VLM在各基准上均稳定优于基线模型,最高提升达12.3%,且优于测试时耦合世界模型的方法,同时避免了昂贵的推理时生成过程。结果表明,世界模型不仅可作推理工具,更可作为高效、可扩展的训练教师,帮助VLM内化空间想象力。

原文摘要 · Abstract (English)

Vision-language models (VLMs) have shown strong performance on static visual understanding, yet they still struggle with dynamic spatial reasoning that requires imagining how scenes evolve under egocentric motion. Recent efforts address this limitation either by scaling spatial supervision with synthetic data or by coupling VLMs with world models at inference time. However, the former often lacks explicit modeling of motion-conditioned state transitions, while the latter incurs substantial computational overhead. In this work, we propose World2VLM, a training framework that distills spatial imagination from a generative world model into a vision-language model. Given an initial observation and a parameterized camera trajectory, we use a view-consistent world model to synthesize geometrically aligned future views and derive structured supervision for both forward (action-to-outcome) and inverse (outcome-to-action) spatial reasoning. We post-train the VLM with a two-stage recipe on a compact dataset generated by this pipeline and evaluate it on multiple spatial reasoning benchmarks. World2VLM delivers consistent improvements over the base model across diverse benchmarks, including SAT-Real, SAT-Synthesized, VSI-Bench, and MindCube. It also outperforms the test-time world-model-coupled methods while eliminating the need for expensive inference-time generation. Our results suggest that world models can serve not only as inference-time tools, but also as effective training-time teachers, enabling VLMs to internalize spatial imagination in a scalable and efficient manner.

空间推理世界模型视觉语言模型知识蒸馏

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。