利用人脸深度图提升伪造视频检测能力,效果显著。
Exploring Depth Information for Detecting Manipulated Face Videos
- 通过深度图变换器逐块估计人脸深度图,捕捉篡改导致的局部异常。
- 设计多头深度注意力机制,融合深度信息与主干特征,提升检测精度。
- 适合关注视频伪造检测、计算机视觉安全的研究者和开发者。
人脸篡改检测因图像/视频的可靠性与安全性问题受到广泛关注。近期研究聚焦于利用辅助信息或先验知识来捕捉鲁棒的篡改痕迹,已显示出良好前景。然而,作为重要的人脸特征之一,深度图在人脸篡改检测领域仍鲜有研究。本文探索将人脸深度图作为辅助信息用于鲁棒的篡改检测。为此,我们提出面部深度图变换器(FDMT),从RGB人脸图像中逐块估计深度图,以捕捉篡改引起的局部深度异常。随后,采用新设计的多头深度注意力(MDA)机制,将估计的深度图与主干特征融合。此外,还提出RGB-深度不一致注意力(RDIA)模块,有效捕获多帧输入间的帧间不一致性。大量实验验证了所提方法在人脸篡改检测上的优势。
原文摘要 · Abstract (English)
Face manipulation detection has been receiving a lot of attention for the reliability and security of the face images/videos. Recent studies focus on using auxiliary information or prior knowledge to capture robust manipulation traces, which are shown to be promising. As one of the important face features, the face depth map, which has shown to be effective in other areas such as face recognition or face detection, is unfortunately paid little attention to in literature for face manipulation detection. In this paper, we explore the possibility of incorporating the face depth map as auxiliary information for robust face manipulation detection. To this end, we first propose a Face Depth Map Transformer (FDMT) to estimate the face depth map patch by patch from an RGB face image, which is able to capture the local depth anomaly created due to manipulation. The estimated face depth map is then considered as auxiliary information to be integrated with the backbone features using a Multi-head Depth Attention (MDA) mechanism that is newly designed. We also propose an RGB-Depth Inconsistency Attention (RDIA) module to effectively capture the inter-frame inconsistency for multi-frame input. Various experiments demonstrate the advantage of our proposed method for face manipulation detection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。