用视觉语言模型指导蒸馏,让小模型高效复现大模型的机器人操作能力。
Offline Semantic Guidance for Efficient Vision-Language-Action Policy Distillation

- 用视觉语言模型提供任务阶段和操作方向等语义监督信号,增强蒸馏效果。
- 学生模型仅158M参数,大小缩小44倍,推理速度提升3.28倍,性能损失仅0.27%。
- 无需依赖大模型即可独立运行,适合部署在资源受限的实时控制场景。
千亿参数的视觉-语言-动作(VLA)策略在机器人操作中表现出色,但其规模和推理开销限制了实时闭环控制的应用。本文提出VLA-AD蒸馏框架,利用视觉语言模型作为离线语义监督器,将大型VLA教师模型压缩为轻量级学生策略。与仅依赖低层动作模仿不同,VLA-AD在训练时引入高阶语义引导信号,包括任务阶段锚点和多帧操作方向描述,这些信号在测试时不再需要。在三个LIBERO基准上评估,以OpenVLA-7B为教师时,生成的158M参数学生模型实现44倍压缩,平均性能相对差距仅0.27%,在RTX 4090上达到12.5 Hz,推理速度比OpenVLA-7B快3.28倍。该方法还适用于π₀.₅-4B教师,在两个数据集上表现优于教师,且在libero_goal上差距不超过0.53%。分析表明,阶段级监督和多帧方向提示可降低学生对教师动作噪声(如高频夹爪误操作)的敏感性。结果证明,来自视觉语言模型的离线语义指导能显著提升VLA策略蒸馏的效率、鲁棒性和可部署性。
原文摘要 · Abstract (English)
Billion-parameter Vision-Language-Action (VLA) policies have recently shown impressive performance in robotic manipulation, yet their size and inference cost remain major obstacles for real-time closed-loop control. We introduce \textbf{VLA-AD}, a distillation framework that uses a Vision-Language Model as an offline semantic supervisor to transfer large VLA teachers into lightweight student policies. Instead of relying only on low-level action imitation, VLA-AD augments teacher-provided 7-DoF action targets with high-level semantic guidance, including task phase anchors and multi-frame operating-direction descriptions. These auxiliary signals are used only during training: at test time, the student policy runs independently, with neither the VLA teacher nor the VLM required. We evaluate VLA-AD on three LIBERO benchmark suites. Using OpenVLA-7B as the teacher, our method produces a 158M-parameter student, yielding a $44\times$ reduction in model size while matching the teacher with only a $0.27\%$ average relative gap. The resulting policy runs at 12.5 Hz on an RTX 4090, achieving a $3.28\times$ inference speedup over OpenVLA-7B. We further show that the same semantic distillation pipeline generalizes to a different $π_{0.5}$-4B teacher, where the student outperforms the teacher on two suites and remains within $0.53\%$ on \texttt{libero\_goal}. Additional analysis indicates that phase-level supervision and multi-frame directional cues make the student less sensitive to noisy teacher actions, such as erroneous high-frequency gripper changes. Overall, VLA-AD demonstrates that offline semantic guidance from VLMs can substantially improve the efficiency, robustness, and deployability of VLA policy distillation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。