用生成模型实时修复受损图像，提升视觉控制鲁棒性。
Generalizable Image Repair for Robust Visual Control
- 结合CycleGAN与pix2pix，实现无配对与有配对图像修复。
- 在模拟自动驾驶场景中，修复后控制成功率提升显著。
- 专为控制性能设计损失函数，修复更贴合下游任务需求。
基于视觉的控制依赖准确感知以实现鲁棒性。然而，传感器噪声、恶劣天气和动态光照引起的图像分布变化会降低感知质量，导致次优控制决策。现有方法如领域自适应和对抗训练虽能提升鲁棒性，但难以泛化到未见过的损坏类型，且计算开销大。为此，我们提出一种实时图像修复模块，在图像被控制器使用前进行恢复。该方法利用生成对抗网络，包括无需配对数据的CycleGAN和有配对数据时的pix2pix，分别应对新出现的损坏和提升修复质量。为确保与控制性能对齐，引入面向控制的损失函数，优先保证修复图像的感知一致性。我们在包含多种视觉退化的模拟自动驾驶赛车环境中评估了该方法，结果表明，相比基线方法，本方案显著提升了控制性能，有效缓解了分布偏移并增强了控制器可靠性。
原文摘要 · Abstract (English)
Vision-based control relies on accurate perception to achieve robustness. However, image distribution changes caused by sensor noise, adverse weather, and dynamic lighting can degrade perception, leading to suboptimal control decisions. Existing approaches, including domain adaptation and adversarial training, improve robustness but struggle to generalize to unseen corruptions while introducing computational overhead. To address this challenge, we propose a real-time image repair module that restores corrupted images before they are used by the controller. Our method leverages generative adversarial models, specifically CycleGAN and pix2pix, for image repair. CycleGAN enables unpaired image-to-image translation to adapt to novel corruptions, while pix2pix exploits paired image data when available to improve the quality. To ensure alignment with control performance, we introduce a control-focused loss function that prioritizes perceptual consistency in repaired images. We evaluated our method in a simulated autonomous racing environment with various visual corruptions. The results show that our approach significantly improves performance compared to baselines, mitigating distribution shift and enhancing controller reliability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。