让机器人操作更快更准,通过视觉自适应调整采样和推理策略。
VADF: Vision-Adaptive Diffusion Policy Framework for Efficient Robotic Manipulation

- 训练时用轻量网络动态评估样本难度,优先学习难样本。
- 推理时按视觉输入拆分任务,简单动作少噪声、复杂动作多迭代。
- 无需改动现有模型,适配任意扩散策略,提升早期成功率。
扩散策略在机器人操作中日益主流,但因均匀采样导致负样本分布不均,缺乏对样本难易度的感知,造成训练收敛慢且推理时常超时失败。本文提出视觉自适应扩散策略框架VADF,一种视觉驱动的双适应架构,显著减少收敛步数并实现推理早期成功。训练阶段引入轻量级MLP-based损失预测器ALN,实时量化每步样本难度,结合硬负样本挖掘进行加权采样,促进自适应权重更新与更快收敛。推理阶段设计分层视觉任务分割器HVTS,根据视觉输入将高层任务指令分解为多阶段低层子指令,通过分配不同噪声调度:简单动作采用短噪声周期与长直接执行序列,复杂动作则用长噪声步数与短执行序列,大幅降低计算开销并显著提升早期成功概率。
原文摘要 · Abstract (English)
Diffusion policies are becoming mainstream in robotic manipulation but suffer from hard negative class imbalance due to uniform sampling and lack of sample difficulty awareness, leading to slow training convergence and frequent inference timeout failures. We propose VADF (Vision-Adaptive Diffusion Policy Framework), a vision-driven dual-adaptive framework that significantly reduces convergence steps and achieves early success in inference, with model-agnostic design enabling seamless integration into any diffusion policy architecture. During training, we introduce Adaptive Loss Network (ALN), a lightweight MLP-based loss predictor that quantifies per-step sample difficulty in real time. Guided by hard negative mining, it performs weighted sampling to prioritize high-loss regions, enabling adaptive weight updates and faster convergence. In inference, we design the Hierarchical Vision Task Segmenter (HVTS), which decomposes high-level task instructions into multi-stage low-level sub-instructions based on visual input. It adaptively segments action sequences into simple and complex subtasks by assigning shorter noise schedules with longer direct execution sequences to simple actions, and longer noise steps with shorter execution sequences to complex ones, thereby dramatically reducing computational overhead and significantly improving the early success rate.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。