用AI生成伪装图像提升目标检测在复杂环境下的鲁棒性
Domain shift-robust object detection with GenAI image editing

- 用扩散模型生成带伪装的合成训练图像,模拟真实域变化
- 在树叶和网状伪装下,检测精度提升20.1和14.4 mAP
- 适合低数据量、高复杂度场景的目标检测任务
目标检测器在光照、天气或遮挡等域偏移下性能显著下降,因其依赖训练数据中的视觉捷径而难以泛化。在数据稀缺的专业场景中,获取足够多的真实域变体样本尤为困难。尽管基于扩散模型的生成编辑技术已能提升检测器的域内性能,其对域外鲁棒性的潜力仍待探索。本文以军用车辆伪装检测为挑战性场景,发现未伪装训练数据在真实测试图像(含树叶、网罩及多光谱伪装)上对15类车辆的检测性能严重退化。我们使用Qwen Image Edit 2509与Flux.2 Dev两个扩散编辑模型,结合微调后的LoRA版本,对训练数据添加伪装。以黑条遮挡作为低质量基线。采用GroundingDINO检测器,在真实与合成数据上训练,生成式伪装增强使树叶和网罩伪装场景的mAP分别提升20.1和14.4;多光谱伪装较难生成,但LoRA微调仍带来4.4 mAP的提升。
原文摘要 · Abstract (English)
Object detectors often degrade under domain shifts such as changes in lighting, weather, or occlusion. These shifts alter object appearance and expose a reliance on visual shortcuts learned from the training distribution that do not generalize across domains. Acquiring sufficient real-world samples to capture such domain variation is particularly difficult in specialized, low-data settings. Recent advances in diffusion-based generative image editing have shown promise for improving the in-domain performance of object detectors through synthetic data augmentation. However, their potential to improve out-of-domain robustness remains largely unexplored. We hypothesize that generative image editing can simulate a controlled domain shift in training data, effectively bridging the gap between source and target domains. To test this, we studied camouflaged military vehicle detection as a challenging domain shift scenario. Detectors trained on uncamouflaged data demonstrate substantial degradation on real test imagery containing foliage, netting, and multi-spectral camouflage across 15 vehicle classes in close-up, ground-level imagery. We used two diffusion-based editing models, Qwen Image Edit 2509 and Flux.2 Dev, to synthetically add camouflage to the training data, alongside a LoRA fine-tuned version of Qwen. A non-generative black-bar occlusion baseline served as a lower bound on augmentation quality. Using a GroundingDINO detector trained on real and synthetic data, generative camouflage augmentation yielded substantial mAP improvements for foliage (+20.1) and netting (+14.4) camouflage. Generating multi-spectral camouflage proved more challenging, but LoRA fine-tuning improved performance by 4.4 mAP over the uncamouflaged baseline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。