arXiv:2606.27504cs.CV2026-06

ReWorld通过优化中间表示,让自动驾驶模型更高效地从环境理解到动作生成。

ReWorld: Representation Learning for World Action Models

论文配图:ReWorld: Representation Learning for World Action Models
图 1 · 摘自论文原文
  • 显式优化视频与动作间的潜在表示路径,提升信息传递效率。
  • nuScenes上FVD降低至61.9,闭环测试中规划性能提升至90.4。
  • 无需外部模型,仅增加0.3%训练成本,适合部署在轻量级自动驾驶系统。

世界动作模型(WAM)将环境未来预测与动作生成统一,但现有方法仅优化最终输出,忽略中间表示。本文提出ReWorld,首个专为自动驾驶WAM设计的表示学习框架。通过三种互补机制:首先,在Video DiT中间状态施加未来预测监督,编码时间动态,实现自引导采样并使收敛速度提升约一倍;其次,对齐Action DiT状态与对应视频读出,确保规划所用表示保留关键世界信息;第三,使用几何相近但得分低的硬负样本塑造动作空间,有效区分专家轨迹与邻近危险路径。所有监督信号均来自WAM自身生成目标和注意力特征,无需外部编码器或教师模型,每步训练成本仅增加0.3%。实验显示,ReWorld在nuScenes上将FVD从81.3降至61.9,在NAVSIM上闭环PDMS从89.1升至90.4(无需强化学习或测试时评分),并在UCF-101动作识别上将冻结线性探测准确率从68.3%提升至80.2%。结果表明,显式优化的表示是将世界知识转化为规划能力的核心。

原文摘要 · Abstract (English)

World Action Models (WAMs) unify future environment prediction with action generation for autonomous driving, yet existing approaches optimize only the final outputs, leaving intermediate representations as incidental byproducts. We present ReWorld, the first representation learning framework specifically designed for autonomous-driving WAMs. ReWorld explicitly optimizes the latent world-to-action pathway through three complementary mechanisms. First, it imposes future-predictive supervision on intermediate Video DiT states to encode temporal scene dynamics, enabling self-guided sampling and a roughly twofold convergence speedup. Second, it aligns Action DiT states with their attended video readouts so that the retrieved world information is retained in the representations used for planning. Third, it shapes the action space using geometrically close yet low-scoring hard negatives to separate the expert trajectory from nearby unsafe alternatives. ReWorld constructs supervision entirely from the WAM's own generation targets and attended features, requiring no external encoders or teacher models and introducing only 0.3% additional per-step training cost. Experiments show that ReWorld reduces FVD from 81.3 to 61.9 on nuScenes, improves closed-loop PDMS from 89.1 to 90.4 on NAVSIM without reinforcement learning or test-time scoring, and increases frozen linear-probe accuracy from 68.3% to 80.2% on UCF-101 action recognition. These results indicate that explicitly optimized representations are central to translating world knowledge into planning capability in WAMs.

自动驾驶表示学习动作生成扩散模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。