让机器人理解动作背后的意图,提升任务成功率。
Act with Intent: Distilling Behavior Intent for Vision-Language-Action Models

- 用冻结的视觉语言模型提取动作意图,指导行动预测。
- 在两个仿真和真实场景中,成功率最高提升12个百分点。
- 适合需要理解任务目标的机器人控制研究者。
视觉-语言-动作(VLA)模型可将多模态信息转化为机器人动作,但其动作解码器仍主要通过行为克隆训练,忽略了行为背后的服务目标。未来监督虽引入轨迹、运动等信号,却仅捕捉具体实现而非共享语义目标。本文提出意图蒸馏(INDI),将行为级意图注入动作解码器。训练时,一个冻结的教师视觉语言模型基于当前观测、指令、粗略动作摘要及执行视频,解析出意图。部署时,VLA模型在中间解码层恢复该多模态意图表示,结合行为展开与达成状态的表示,组织动作预测。在SimplerEnv-Bridge上,GR00T-N1.7从64.3%提升至84.7%;在RoboCasa Kitchen上,从64.1%提升至70.3%,π₀.₅指标持续提升。真实任务中,平均成功率由62.0%增至68.7%,长周期任务最高提升12.0个百分点。分析表明,恢复的潜在表示被解码器使用,能捕捉行为目标与执行进度,并以目标相关方式组织下游预测。结果表明,显式建模行为语义目标显著提升动作解码性能。
原文摘要 · Abstract (English)
Vision-Language-Action (VLA) models can turn multimodal context into robot actions, but their action decoders are still trained largely by behavior cloning. This supervises which motor command was demonstrated while leaving implicit the local objective served by the behavior under the instruction. Future-based supervision enriches action learning with frames, latent observations, trajectories, or motion representations, but these signals capture particular realizations of what may happen rather than the shared semantic objective of the forthcoming behavior. We propose Intention Distillation (INDI), which distills behavior-level intent into the action decoder. During training, a frozen teacher VLM interprets a demonstrated segment from the current observation, instruction, coarse action summary, and corresponding execution video. From its standard inputs, the deployed VLA recovers the resulting multimodal intent representation at an intermediate decoder layer and uses it to organize action prediction together with representations of how the behavior unfolds and what it achieves. On SimplerEnv-Bridge, INDI improves GR00T-N1.7 from 64.3% to 84.7%, and on RoboCasa Kitchen it improves the controlled GR00T-N1.7 baseline from 64.1% to 70.3%, with consistent gains on $π_{0.5}$ across both benchmarks. In real-world tasks, INDI improves average success from 62.0% to 68.7%, with gains of up to 12.0 pp on longer-horizon tasks. Further analyses show that the recovered latent is used by the decoder, captures behavior objective and execution progress, and organizes downstream predictions in an objective-dependent manner. These results show that action decoders benefit from explicitly modeling the semantic objective of the behavior they generate.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。