让自动驾驶模型先‘想’未来再行动,提升决策智能与效率。
X-Mind: Efficient Visual Chain-of-Thought via Predictive World Model for End-to-End Driving

- 用抽象视觉思维图替代密集帧预测,压缩推理内容至96个符号。
- 通过递归扩散机制,将多步去噪融合进单次前向传播,降低延迟。
- 适合部署在资源受限车载平台的高智能端到端驾驶系统。
预测未来状态对自主代理至关重要,但现有视觉-语言-动作(VLA)模型主要依赖反应式感知-动作映射,缺乏前瞻能力。为弥补这一缺陷,本文提出X-Mind框架,将预测世界模型(PWMs)内化为视觉思维链(Visual CoT),强制模型在执行动作前先模拟未来演化,从而生成基于环境动态、预见行动后果的驾驶策略。为解决效率问题,首先引入紧凑的视觉思维表示:融合鸟瞰图布局与驾驶先验(如导航意图、交通规则)的抽象草图;结合深度压缩自编码器(DC-AE),12帧未来预测被压缩至仅96个令牌,缓解长上下文计算瓶颈。其次,提出递归块扩散方案,将去噪步骤沿大模型层级展开,实现迭代优化的一次前向传播。在大规模真实数据上训练并验证,X-Mind达到有竞争力的端到端驾驶性能,是一种高效低延迟、可直接部署于资源受限车端平台的大规模认知推理解决方案。
原文摘要 · Abstract (English)
Predicting future states is essential for autonomous agents, yet current Vision-Language-Action (VLA) models fundamentally lack this capability, relying instead on reactive perception-action mapping. While integrating Predictive World Models (PWMs) addresses this gap, existing approaches either incur prohibitive cascaded latency or act as shallow terminal tasks that fail to deeply embed forward-looking reasoning. To endow VLA models with this reasoning capability, we propose X-Mind. Rather than treating PWMs as an external auxiliary module, this framework internalizes them as the Visual Chain-of-Thought (Visual CoT). By enforcing a world rollout prior to action, the model is constrained to imagine future evolution first, yielding a driving policy that is robustly grounded in environmental dynamics and aware of the future consequences its actions will unfold. The challenge here is efficiency, and we tackle it on two fronts. First, we introduce a compact representation of visual thinking: an abstract sketch that fuses a Bird's-Eye-View (BEV) layout with abstract driving priors (e.g., navigation intents and traffic rules). Rather than rolling out dense future frames, the model reasons over this sketch as a mental canvas; aided by a Deep Compression Autoencoder (DC-AE), a 12-frame future rollout is reduced to merely 96 tokens, alleviating the long-context computational bottleneck. Second, to accelerate generation further, we propose a recurrent block diffusion scheme that unrolls the denoising steps across the layers of the large drive model, folding iterative refinement into the backbone's one forward pass. Trained and validated on large-scale real-world data, X-Mind achieves competitive end-to-end driving performance, which makes it a highly practical, low-latency solution that successfully deploys large-scale cognitive reasoning directly onto resource-constrained vehicle platforms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。