通过融合RGB与3D面部特征提升深度伪造检测精度
M3D-Net: Multi-Modal 3D Facial Feature Reconstruction Network for Deepfake Detection
- 双流网络从单张图片重建3D面部几何与反光属性
- 在多个数据集上准确率超越现有方法，泛化性强
- 适合关注多模态融合与真实感伪造检测的研究者
随着深度学习在图像生成领域的快速发展，人脸伪造技术已达到前所未有的逼真程度，严重威胁网络安全与信息真实性。现有大多数深度伪造检测方法仅重建孤立的面部属性，未能充分利用多模态特征的互补性。为此，本文提出一种新型多模态3D面部特征重建网络（M3D-Net）。该方法采用端到端双流架构，通过自监督3D面部重建模块从单视角RGB图像中恢复精细的面部几何与反射特性。网络进一步引入3D特征预融合模块（PFM）以自适应调整多尺度特征，并设计多模态融合模块（MFM），利用注意力机制有效整合RGB与3D重建特征。在多个公开数据集上的大量实验表明，本方法在检测准确率与鲁棒性方面均达到当前最优水平，显著优于现有方法，并展现出强大的跨场景泛化能力。
原文摘要 · Abstract (English)
With the rapid advancement of deep learning in image generation, facial forgery techniques have achieved unprecedented realism, posing serious threats to cybersecurity and information authenticity. Most existing deepfake detection approaches rely on the reconstruction of isolated facial attributes without fully exploiting the complementary nature of multi-modal feature representations. To address these challenges, this paper proposes a novel Multi-Modal 3D Facial Feature Reconstruction Network (M3D-Net) for deepfake detection. Our method leverages an end-to-end dual-stream architecture that reconstructs fine-grained facial geometry and reflectance properties from single-view RGB images via a self-supervised 3D facial reconstruction module. The network further enhances detection performance through a 3D Feature Pre-fusion Module (PFM), which adaptively adjusts multi-scale features, and a Multi-modal Fusion Module (MFM) that effectively integrates RGB and 3D-reconstructed features using attention mechanisms. Extensive experiments on multiple public datasets demonstrate that our approach achieves state-of-the-art performance in terms of detection accuracy and robustness, significantly outperforming existing methods while exhibiting strong generalization across diverse scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。