让机器人预测场景变化位置与方式,提升操控泛化能力。
Bridge-WA: Predicting Where and How the World Changes for Robotic Action

- 用三个轻量先验捕捉未来变化:目标状态、干预支持、局部运动方向。
- 在多个数据集和真实机器人上,成功率与鲁棒性显著提升,尤其应对视觉分布外变化。
- 无需生成密集未来图像,有效抑制背景等干扰因素,适合部署于实际机器人系统。
通用视觉-语言-动作模型依赖大规模视觉-语言先验,但有效操作还需预测与动作相关的场景变化。现有世界-动作模型通常依赖大型生成式世界模型或密集未来推演,成本高且浪费算力在与控制弱相关的视觉细节上。我们提出 Bridge-WA,一个轻量级世界-动作框架,将冻结的未来变化教师模型提炼为三个紧凑先验:未来标记(用于预期结果)、变化图(用于干预支持)、运动流图(用于局部转移方向)。一个 WorldBridge 通过多源注意力记忆和时空偏置,将这些先验融入动作变换器,推理时移除教师模型。在 VLABench、RoboTwin2.0、LIBERO-Plus 及真实机器人评估中,Bridge-WA 提升了任务成功率、进展度和鲁棒性,尤其在分布外视觉变化下表现更优。通过聚焦于场景变化的位置与方式,Bridge-WA 抑制了背景、光照、干扰物等无关外观因素,实现无需部署时密集未来图像生成的更好泛化。代码与可视化见:https://hcplab-sysu.github.io/BRIDGE-WA。
原文摘要 · Abstract (English)
General-purpose vision-language-action models benefit from large vision-language priors, but effective manipulation also requires anticipating action-relevant scene changes. Existing world-action models often rely on large generative world models or dense future rollouts, which are expensive and spend capacity on visual details weakly coupled to control. We present Bridge-WA, a lightweight world-action framework that distills a frozen future-change teacher into three compact priors: future tokens for intended outcomes, change maps for intervention support, and motion-flow maps for local transition direction. A WorldBridge conditions the action transformer on these priors through multi-source attention memories and spatial-temporal biases, while the teacher model is removed at inference. Across VLABench, RoboTwin2.0, LIBERO-Plus and real-robot evaluations, Bridge-WA improves task success, progress, and robustness, with particularly clear gains under out-of-distribution visual shifts. By focusing action generation on where and how the scene will change, Bridge-WA suppresses nuisance appearance factors such as background, lighting, and distractors, leading to better generalization without deployment-time dense future-image generation. Code and visualizations are available at: https://hcplab-sysu.github.io/BRIDGE-WA .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。