用注意力机制修复戴口罩人脸,保持身份一致且细节逼真。
Face Mask Removal with Region-attentive Face Inpainting
- 设计多尺度通道-空间注意力模块,增强遮挡区域特征提取。
- 在掩码区域施加监督信号,提升关键区域重建精度。
- 自建含5类口罩的面部数据集,适用于图像编辑与识别优化。
新冠疫情中,口罩普遍佩戴导致人脸识别模型失效。为改善社交互动与图像视频编辑,本文提出一种生成式人脸修复方法,旨在高保真恢复被遮挡面部区域。相比传统修复,人脸修复需兼顾身份一致性与细节真实感。提出多尺度通道-空间注意力模块(M-CSAM),缓解空间信息丢失并学习通道间与通道内相关性;引入仅对掩码区域施加监督信号的新策略,强化关键区域重建。同时,基于CelebA数据集构建包含外科口罩、普通口罩及围巾等五类遮挡物的自定义掩码人脸数据集,覆盖颈部区域。实验表明,所提方法在结构相似性(SSIM)、峰值信噪比(PSNR)和L1损失上优于多个基线模型,视觉效果更佳。代码将公开于GitHub。
原文摘要 · Abstract (English)
During the COVID-19 pandemic, face masks have become ubiquitous in our lives. Face masks can cause some face recognition models to fail since they cover significant portion of a face. In addition, removing face masks from captured images or videos can be desirable, e.g., for better social interaction and for image/video editing and enhancement purposes. Hence, we propose a generative face inpainting method to effectively recover/reconstruct the masked part of a face. Face inpainting is more challenging compared to traditional inpainting, since it requires high fidelity while maintaining the identity at the same time. Our proposed method includes a Multi-scale Channel-Spatial Attention Module (M-CSAM) to mitigate the spatial information loss and learn the inter- and intra-channel correlation. In addition, we introduce an approach enforcing the supervised signal to focus on masked regions instead of the whole image. We also synthesize our own Masked-Faces dataset from the CelebA dataset by incorporating five different types of face masks, including surgical mask, regular mask and scarves, which also cover the neck area. The experimental results show that our proposed method outperforms different baselines in terms of structural similarity index measure, peak signal-to-noise ratio and l1 loss, while also providing better outputs qualitatively. The code will be made publicly available. Code is available at GitHub.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。