arXiv:2506.17592cs.CV2025-06被引 2

SELFI动态调节身份特征,提升深伪检测泛化能力。

SELFI: Selective Fusion of Identity for Generalizable Deepfake Detection

  • 根据样本相关性动态融合身份与视觉特征
  • 跨篡改方法检测平均提升3.1% AUC,DFDC上领先6%
  • 适合需要强泛化性的深伪检测场景

面部身份是深伪检测的有力信号。先前研究发现,即使未显式建模,分类器也会隐式学习身份特征。这导致两种观点冲突:一者抑制身份线索以降低偏差,另一者则将其作为取证证据。为调和分歧,我们验证两个假设:(1) 身份特征是否足以区分深伪;(2) 此类特征在不同篡改方法间是否泛化差。实验表明,身份信息虽有用但具上下文依赖性。某些篡改保留一致的痕迹,而另一些则破坏身份线索,损害泛化能力。因此,身份特征不应被盲目抑制或依赖,而应显式建模并按样本相关性自适应控制。我们提出SELFI(Selective Fusion of Identity),一个可泛化的检测框架,通过动态调节身份使用实现优化。SELFI包含:(1) 针对伪造感知的身份适配器(FAIA),从冻结的人脸识别模型提取身份嵌入,并通过辅助监督投影至伪造相关空间;(2) 身份感知融合模块(IAFM),基于相关性引导机制选择性融合身份与视觉特征。在四个基准上的实验表明,SELFI提升了跨篡改方法的泛化性能,平均优于现有方法3.1% AUC;在具有挑战性的DFDC数据集上,超越前人最佳结果6%。代码将在论文接受后公开。

原文摘要 · Abstract (English)

Face identity provides a powerful signal for deepfake detection. Prior studies show that even when not explicitly modeled, classifiers often learn identity features implicitly. This has led to conflicting views: some suppress identity cues to reduce bias, while others rely on them as forensic evidence. To reconcile these views, we analyze two hypotheses: (1) whether face identity alone is discriminative for detecting deepfakes, and (2) whether such identity features generalize poorly across manipulation methods. Our experiments confirm that identity is informative but context-dependent. While some manipulations preserve identity-consistent artifacts, others distort identity cues and harm generalization. We argue that identity features should neither be blindly suppressed nor relied upon, but instead be explicitly modeled and adaptively controlled based on per-sample relevance. We propose \textbf{SELFI} (\textbf{SEL}ective \textbf{F}usion of \textbf{I}dentity), a generalizable detection framework that dynamically modulates identity usage. SELFI consists of: (1) a Forgery-Aware Identity Adapter (FAIA) that extracts identity embeddings from a frozen face recognition model and projects them into a forgery-relevant space via auxiliary supervision; and (2) an Identity-Aware Fusion Module (IAFM) that selectively integrates identity and visual features using a relevance-guided fusion mechanism. Experiments on four benchmarks show that SELFI improves cross-manipulation generalization, outperforming prior methods by an average of 3.1\% AUC. On the challenging DFDC dataset, SELFI exceeds the previous best by 6\%. Code will be released upon paper acceptance.

深伪检测身份特征泛化能力自适应融合

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。