用语义蒸馏让机器人理解跨阶段动作，提升复杂操作成功率。
CASD: Chunk-Aligned Semantic Distillation for Multi-StageRobot Manipulation
- 通过视觉语言模型分割演示，为整段动作生成加权语义目标。
- 在LIBERO和RoboTwin上分别实现98.9%和93.0%的成功率，优于基线。
- 无需实时调用VLM，适合部署在资源受限的机器人系统中。
一个动作片段可能跨越多个操作阶段，但其首个步骤的标签仅描述当前阶段。我们提出块对齐语义蒸馏（CASD），为整个动作片段生成语义目标。离线视觉-语言模型将示范分割为描述性阶段，各阶段在动作片段中的占比决定加权语义目标，包含阶段间过渡。一个CASD生成器从当前观测、机器人状态和任务指令中学习预测该目标。随后冻结生成器，训练以预测结果为条件的策略。语义分支每策略查询运行一次，无需在线VLM调用或推理轨迹解码。在标注的LIBERO训练样本上，单阶段与跨边界片段的教师匹配均显著高于随机水平。我们在三个Fast-WAM变体及DreamZero集成上评估，覆盖四个基准，包括LIBERO-Plus上的分布外情况。相比已有参考，IDM+CASD在LIBERO上平均成功率达98.9%（参考为98.0%），而Uncond低于其参考值。Joint+CASD在RoboTwin 2.0上达93.0%（参考90.6%），DreamZero+CASD在四类MolmoSpaces操作上平均达47.9%（参考40.7%）。性能随主干网络集成不同而变化。
原文摘要 · Abstract (English)
An action chunk can span several stages of a manipulation task, yet a label for its first step describes only the current stage. We introduce Chunk-Aligned Semantic Distillation (CASD), which derives semantic targets for entire action chunks. An offline vision--language model segments demonstrations into described stages. Their occupancy within each action chunk determines a weighted semantic target, including transitions between stages. A CASD generator learns to predict this target from the current observation, robot state, and task instruction. We then freeze the generator and train a policy conditioned on its predictions. The semantic branch runs once per policy query, without online VLM calls or reasoning-trace decoding. Teacher matching on annotated LIBERO training episodes is above chance for both single-stage and boundary-crossing chunks. We evaluate three Fast-WAM variants and a DreamZero integration across four benchmarks, including distribution shifts on LIBERO-Plus. Compared with published references, IDM+CASD reaches 98.9\% versus 98.0\% average success on LIBERO, while Uncond falls below its reference. Joint+CASD reaches 93.0\% versus 90.6\% on RoboTwin 2.0, and DreamZero+CASD reaches a 47.9\% four-category MolmoSpaces manipulation average versus 40.7\%. Performance varies across backbone integrations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。