arXiv:2409.16945cs.CV2024-09TPAMI被引 3

为提升伪造人脸检测泛化能力,提出专用预训练模型与竞争性微调方法。

Revisiting Face Forgery Detection: From Facial Representation to Forgery Detection

论文配图:Revisiting Face Forgery Detection: From Facial Representation to Forgery Detection
图 1 · 摘自论文原文
  • 用真实人脸自监督预训练构建专用面部表征模型
  • 通过竞争学习机制激发模型识别隐含伪造痕迹,提升检测效果
  • 引入置信度阈值优化,增强推理可靠性,适合安全敏感场景

人脸伪造检测(FFD)旨在判断数字人脸是否为真实。由于不同生成算法导致的伪造模式差异,现有方法常在训练数据上过拟合,泛化能力差。当前方法多依赖通用图像预训练主干网络进行微调,但缺乏面部领域知识,难以捕捉复杂面部特征,限制了对隐含伪造线索的识别与泛化。为此,本文从预训练到微调全流程重构FFD流程,提出一种基于真实人脸自监督预训练的专用骨干网络,具备更强面部表征能力;进一步设计竞争性微调框架,通过竞争学习机制促使模型主动发现隐含伪造线索;同时提出置信度阈值优化机制,提升推理可靠性。大量实验证明,该方法在人脸伪造检测及展示攻击检测等任务中表现优异。代码与模型已开源。

原文摘要 · Abstract (English)

Face Forgery Detection (FFD), or Deepfake detection, aims to determine whether a digital face is real or fake. Due to different face synthesis algorithms with diverse forgery patterns, FFD models often overfit specific patterns in training datasets, resulting in poor generalization to other unseen forgeries. Existing FFD methods primarily leverage pre-trained backbones with general image representation capabilities and fine-tune them to identify facial forgery cues. However, these backbones lack domain-specific facial knowledge and insufficiently capture complex facial features, thus hindering effective implicit forgery cue identification and limiting generalization. Therefore, it is essential to revisit FFD workflow across the \textit{pre-training} and \textit{fine-tuning} stages, achieving an elaborate integration from facial representation to forgery detection to improve generalization. Specifically, we develop an FFD-specific pre-trained backbone with superior facial representation capabilities through self-supervised pre-training on real faces. We then propose a competitive fine-tuning framework that stimulates the backbone to identify implicit forgery cues through a competitive learning mechanism. Moreover, we devise a threshold optimization mechanism that utilizes prediction confidence to improve the inference reliability. Comprehensive experiments demonstrate that our method achieves excellent performance in FFD and extra face-related tasks, \ie, presentation attack detection. Code and models are available at \href{https://github.com/zhenglab/FFDBackbone}{https://github.com/zhenglab/FFDBackbone}.

伪造检测面部表征自监督泛化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。