让视觉语言动作模型在延迟下仍能平滑异步执行
Action ControlNet: A Lightweight Delay-Aware Adapter for Smooth Asynchronous Control in Vision-Language-Action Models
- 用执行过的运动残差作为条件，动态修正动作预测
- 在真实机器人上降低动作抖动，提升接触操作稳定性
- 轻量级插件式设计，无需重训练或改架构
视觉-语言-动作（VLA）模型在通用机器人操作中展现出巨大潜力，但推理延迟仍是高频稳定控制的主要障碍。异步执行通过重叠策略推理与动作执行缓解该瓶颈，但此时下一动作块仍基于过时观测预测，导致动作交接不连续、抖动及接触丰富操作失败。现有方法通常需完整重训练或特定运行时逻辑。本文提出轻量级延迟感知适配器Action ControlNet（ACNet），利用已执行运动的后缀作为残差条件，对大部分冻结的动作头进行修正。ACNet保持预训练主干不变，引入极少可训练参数，兼容扩散和流匹配等生成式动作头。在Kinetix、Meta-World MT50及真实SO-ARM101平台测试中，ACNet在推理延迟下显著提升鲁棒性，轨迹更平滑，且比全延迟条件重训练更轻量。
原文摘要 · Abstract (English)
Vision-language-action (VLA) models have shown strong potential for general-purpose robot manipulation, but their inference latency remains a major obstacle to stable high-frequency control. Asynchronous execution mitigates this bottleneck by overlapping policy inference with action execution, yet the next action chunk is still predicted from stale observations while the robot continues to move. Direct chunk stitching therefore introduces handoff discontinuities, action jitter, and failures in contact-rich manipulation. Existing remedies typically require either full-policy retraining or architecture-specific runtime logic. This work proposes Action ControlNet (ACNet), a lightweight delay-aware adapter that uses the executed motion suffix as a residual condition for a mostly frozen action head. ACNet leaves the pretrained backbone unchanged, introduces few trainable parameters, and remains compatible with generative action heads such as diffusion and flow matching. On Kinetix, Meta-World MT50, and a real-world SO-ARM101 platform, ACNet improves robustness under inference delay and yields smoother asynchronous trajectories than direct chunk stitching, while remaining more lightweight than full delay-conditioned retraining.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。