让机器人能自动判断任务完成,实现复杂操作的稳定执行。
SeqVLA: Sequential Task Execution for Long-Horizon Manipulation with Completion-Aware Vision-Language-Action Model
- 在π₀模型基础上增加任务完成检测头,实现动作与状态感知协同。
- 在7步沙拉打包和4步糖果打包任务中成功率达90%以上,显著优于基线。
- 适合需要长时间、多步骤操作的机器人系统,如智能仓储与家庭服务。
长时程机器人操作需按严格顺序执行多个相互依赖的子任务,而子任务完成状态检测错误会导致后续失败。现有视觉-语言-动作(VLA)模型如π₀虽擅长连续低层控制,但缺乏内部信号判断子任务是否完成,因此在序列场景中表现脆弱。本文提出SeqVLA,作为π₀的增强版,通过为每个子任务添加轻量级完成检测头,使模型既能生成操作动作,又能自主触发子任务间切换。研究了四种微调策略,包括联合/分步微调以及全量/冻结主干网络。在两个多阶段任务上测试:含7个子任务的沙拉打包与含4个子任务的糖果打包。结果表明,SeqVLA显著优于π₀及其他强基线,在整体成功率上提升明显;其中联合微调且不冻结主干的方案表现最优,完成度预测最准确,彻底消除序列相关失败,实现鲁棒的长时程执行。研究证实,将动作生成与子任务感知结合对可扩展的序列操作至关重要。
原文摘要 · Abstract (English)
Long-horizon robotic manipulation tasks require executing multiple interdependent subtasks in strict sequence, where errors in detecting subtask completion can cascade into downstream failures. Existing Vision-Language-Action (VLA) models such as $π_0$ excel at continuous low-level control but lack an internal signal for identifying when a subtask has finished, making them brittle in sequential settings. We propose SeqVLA, a completion-aware extension of $π_0$ that augments the base architecture with a lightweight detection head perceiving whether the current subtask is complete. This dual-head design enables SeqVLA not only to generate manipulation actions but also to autonomously trigger transitions between subtasks. We investigate four finetuning strategies that vary in how the action and detection heads are optimized (joint vs. sequential finetuning) and how pretrained knowledge is preserved (full finetuning vs. frozen backbone). Experiments are performed on two multi-stage tasks: salad packing with seven distinct subtasks and candy packing with four distinct subtasks. Results show that SeqVLA significantly outperforms the baseline $π_0$ and other strong baselines in overall success rate. In particular, joint finetuning with an unfrozen backbone yields the most decisive and statistically reliable completion predictions, eliminating sequence-related failures and enabling robust long-horizon execution. Our results highlight the importance of coupling action generation with subtask-aware detection for scalable sequential manipulation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。