用注意力引导的增强方法,让机器人在视觉变化下仍能稳定学习行为。
Enhancing Visual Domain Robustness in Behaviour Cloning via Saliency-Guided Augmentation

- 基于策略注意力动态调节图像增强强度,只在无关区域激进处理。
- 在模拟和真实场景中均显著提升对光影、背景、干扰物变化的鲁棒性。
- 无需修改模型结构或额外训练,可直接嵌入现有行为克隆框架。
在基于视觉的行为克隆(BC)中,传统的随机裁剪和颜色抖动等图像增强方法在显著的视觉域偏移(如阴影、干扰物、背景变化)下表现不佳。基于叠加的增强方法虽在计算机视觉中展现潜力,但其在行为克隆中的适用性存疑,因需保留任务关键语义、时空关系及智能体-目标交互。为此,本文提出面向视觉行为克隆的显着性引导增强方法 RoboSaGA,属于叠加类增强。RoboSaGA 利用策略驱动的显着性,在像素级动态调整增强强度,仅在任务无关区域进行强增强,同时保留关键信息。该方法可无缝集成至现有架构,无需结构改动或额外学习目标。实验表明,RoboSaGA 在保持域内性能的同时,显著提升对视觉域偏移的鲁棒性,包括干扰物与背景变化、光照与阴影变化。代码已公开于 https://github.com/Zheyu-Zhuang/RoboSaGA。
原文摘要 · Abstract (English)
In vision-based behavior cloning (BC), conventional image augmentations such as Random Crop and Color Jitter often fall short under substantial visual domain shifts, including changes in shadows, distractors, and backgrounds. Superimposition-based augmentations, which blend in-domain and out-of-domain images, have shown promise for improving generalization in computer vision, but their suitability for BC remains uncertain because task-critical semantics, spatiotemporal relationships, and agent-target interactions must be preserved. To address this, we introduce RoboSaGA, a Saliency-Guided Augmentation method within the superimposition family tailored for vision-based BC. RoboSaGA dynamically adjusts augmentation intensity at the pixel level using policy-driven saliency, enabling aggressive augmentation in task-irrelevant regions while preserving task-critical information. It integrates seamlessly into existing architectures without requiring structural modifications or additional learning objectives. Experiments in both simulated and real-world settings show that RoboSaGA preserves in-domain performance while substantially improving robustness to visual domain shifts, including distractor and background changes, as well as lighting and shadow variations. Code is available at https://github.com/Zheyu-Zhuang/RoboSaGA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。