让机器人双手操作更顺滑，人类干预时自动避免动作突变。
Hand-in-the-Loop: Improving VLA Policies for Dexterous Manipulation via Seamless Hand-Arm Intervention
- 人类纠正时融合意图与自主策略，防止手部动作突然跳变。
- 干预抖动降低99.8%，抓取失败减少87.5%，任务完成快19.1%。
- 适合需要精细双手协作的机器人操作场景，提升人机协同效率。
视觉-语言-动作（VLA）模型在灵巧操作中易因高维动作空间和接触密集动态导致误差累积。尽管交互式模仿学习（IIL）可通过人工修正数据优化策略，但高自由度机械手在干预时刻的人机指令不匹配仍引发动作突变（‘手势跳跃’）。本文提出手-臂协同干预方法HandITL，通过无缝融合人类修正意图与自主策略，避免双臂灵巧操作中的动作突变。相比直接遥控，HandITL将干预抖动降低99.8%，抓取失败减少87.5%，平均完成时间缩短19.1%。在需双臂协调、工具使用及长时程精细操作的任务中验证有效。用于收集修正数据以优化策略时，其训练出的模型在三个长时程灵巧任务上平均性能优于标准遥控数据训练模型19%。
原文摘要 · Abstract (English)
Vision-Language-Action (VLA) models are prone to compounding errors in dexterous manipulation, where high-dimensional action spaces and contact-rich dynamics amplify small policy deviations over long horizons. While Interactive Imitation Learning (IIL) can refine policies through human correction data, applying it to high-degree-of-freedom (DoF) robotic hands remains challenging due to a command mismatch between human teleoperation and policy execution at the intervention moment, which causes abrupt robot-hand configuration changes, or "gesture jumps". We present Hand-in-the-Loop (HandITL), a seamless human-in-the-loop intervention method that blends human corrective intent with autonomous policy execution to avoid gesture jumps during bimanual dexterous manipulation. Compared with taking over control using direct teleoperation, HandITL reduces intervention jitter by 99.8% and preserves robust post-intervention manipulation, reducing grasp failures by 87.5% and mean completion time by 19.1%. We validate HandITL on tasks requiring bimanual coordination, tool use, and fine-grained long-horizon manipulation. When used to collect correction data for policy refinement, HandITL yields policies that outperform those trained with standard teleoperation data by 19% on average across three long-horizon dexterous tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。