无需微调，用想象推演让视觉语言动作模型更听话、更靠谱。
DREAMSTEER: Latent World Models Can Steer VLA Policies During Deployment Without Any Finetuning
- 用潜空间世界模型和价值模型，在部署时推理并筛选动作序列。
- 在真实机器人任务中，成功率从23.75%提升至66.25%，指令遵循率从38.75%升至56.25%。
- 适合部署阶段应对未知环境变化，尤其适用于无法重新训练的场景。
预训练的视觉-语言-动作（VLA）策略展现出良好的零样本泛化能力，但在部署时遭遇分布偏移，导致鲁棒性下降和指令遵循不一致。以往方法通常依赖目标环境中的示范数据进行微调，但此假设难以满足。本文提出DREAMSTEER，一种无需微调或参数修改的部署期策略引导框架。核心思想是利用潜空间世界模型与价值模型对预训练VLA策略进行引导。部署时，从VLA策略和预定义运动基元中采样候选动作片段，通过条件于动作的潜空间世界模型预测其结果，并由语言条件价值模型对想象轨迹进行排序。在四个包含未知物体的真实世界操作基准上，相较基础VLA策略，任务成功率从23.75%提升至66.25%，指令遵循准确率从38.75%提升至56.25%。
原文摘要 · Abstract (English)
Pretrained vision-language-action (VLA) policies show promising zero-shot generalization, but often fail under deployment-time distribution shift, leading to decreased robustness and inconsistent instruction following. While prior work commonly tackles this by finetuning on in-distribution data, it assumes demonstrations collected on tasks in the target environment. In this work, we propose DREAMSTEER, a deployment-time steering framework for pretrained VLAs without any finetuning or parameter modifications. The key insight in DREAMSTEER is to leverage a latent world model and a value model to steer pretrained VLA policies. During deployment, DREAMSTEER samples candidate action chunks from a VLA policy and predefined motion primitives, imagines their outcomes using an action-conditioned latent world model, and ranks the imagined trajectories with a language-conditioned value model. Across four real-world manipulation benchmarks with unseen objects, DREAMSTEER improves task success rate from 23.75% to 66.25% and instruction-following accuracy from 38.75% to 56.25% over the base VLA policy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。