通过渐进式训练，让视觉语言模型摆脱时空幻觉，提升动态推理能力。
A Progressive Training Strategy for Vision-Language Models to Counteract Spatio-Temporal Hallucinations in Embodied Reasoning
- 构建分解时空步骤的思维链数据集，引导模型建立逻辑结构。
- 先用标注数据预训练，再用弱标注数据微调，性能差距缩小至6.53%。
- 适合需要真实动态推理的机器人、自动驾驶等场景研究者。
视觉语言模型在静态图像理解上已取得显著进展，但在时空推理方面仍面临重大挑战。主要瓶颈是‘多图推理幻觉’：正向与反向时间查询间性能下降超过70%，暴露了模型依赖表面捷径而非真实因果理解的问题。为此，我们首先构建了一个新的思维链（CoT）数据集，将复杂推理拆解为详细的时空步骤和确定性判断。在此基础上，提出一种渐进式训练框架：先在该CoT数据集上进行监督预训练以建立逻辑结构，再利用可扩展的弱标签数据进行微调以增强泛化能力。实验表明，该方法不仅提升了基础模型准确率，还将正向-反向性能差距从超过70%降至6.53%，证实其能有效发展真实动态推理能力，并减少当前VLM固有的时间偏差。
原文摘要 · Abstract (English)
Vision-Language Models (VLMs) have made significant strides in static image understanding but continue to face critical hurdles in spatiotemporal reasoning. A major bottleneck is "multi-image reasoning hallucination", where a massive performance drop between forward and reverse temporal queries reveals a dependence on superficial shortcuts instead of genuine causal understanding. To mitigate this, we first develop a new Chain-of-Thought (CoT) dataset that decomposes intricate reasoning into detailed spatiotemporal steps and definitive judgments. Building on this, we present a progressive training framework: it initiates with supervised pre-training on our CoT dataset to instill logical structures, followed by fine-tuning with scalable weakly-labeled data for broader generalization. Our experiments demonstrate that this approach not only improves backbone accuracy but also slashes the forward-backward performance gap from over 70\% to only 6.53\%. This confirms the method's ability to develop authentic dynamic reasoning and reduce the inherent temporal biases of current VLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。