用GAN从单张图自动去除遮挡,还原真实场景细节。
Deep Generative Adversarial Network for Occlusion Removal from a Single Image
- 两阶段CNN先分割遮挡物,再用GAN生成逼真内容填补
- 在自建遮挡数据集上实现零样本泛化,效果优于现有方法
- 适合图像修复、安防监控等需去遮挡的场景
如今,廉价成像设备的发展使得多媒体内容在网络上的获取与分享大幅增加。尽管成像传感器技术不断进步,但遮挡等问题仍影响拍摄质量,并降低监控、检测与识别等应用性能。遮挡分割因尺度变化、光照差异等难题而困难,从前景遮挡中恢复场景也面临挑战,尤其需要准确估计被遮区域并保持上下文一致性。图像去围栏任务尤为复杂,因遮挡物形状、纹理、颜色、图案多样且环境常杂乱。本文提出一种全自动两阶段卷积神经网络,用于围栏类遮挡的分割与缺失内容补全。通过生成对抗网络(GAN)在单次输入中合成结构与纹理均真实的视觉内容以实现图像修复。为评估零样本泛化能力,我们在自建的围栏类遮挡分割数据集上测试了训练后的遮挡检测模型。该数据集已开源于GitHub。
原文摘要 · Abstract (English)
Nowadays, the enhanced capabilities of in-expensive imaging devices have led to a tremendous increase in the acquisition and sharing of multimedia content over the Internet. Despite advances in imaging sensor technology, annoying conditions like \textit{occlusions} hamper photography and may deteriorate the performance of applications such as surveillance, detection, and recognition. Occlusion segmentation is difficult because of scale variations, illumination changes, and so on. Similarly, recovering a scene from foreground occlusions also poses significant challenges due to the complexity of accurately estimating the occluded regions and maintaining coherence with the surrounding context. In particular, image de-fencing presents its own set of challenges because of the diverse variations in shape, texture, color, patterns, and the often cluttered environment. This study focuses on the automatic detection and removal of occlusions from a single image. We propose a fully automatic, two-stage convolutional neural network for fence segmentation and occlusion completion. We leverage generative adversarial networks (GANs) to synthesize realistic content, including both structure and texture, in a single shot for inpainting. To assess zero-shot generalization, we evaluated our trained occlusion detection model on our proposed fence-like occlusion segmentation dataset. The dataset can be found on GitHub.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。