提出可扩展的面部安全预训练模型,统一检测深度伪造、扩散生成和欺骗攻击。
Scalable Face Security Vision Foundation Model for Deepfake, Diffusion, and Spoofing Detection
- 设计三重学习目标3C,结合掩码图像建模与实例判别,捕捉人脸局部与全局特征。
- 在11个公开数据集上超越多种主流视觉模型,跨域检测性能显著提升。
- 适合需要高效通用人脸安全检测的开发者与研究者使用。
面对海量无标签真实人脸数据,如何学习鲁棒且可迁移的人脸表征以提升各类人脸安全任务的泛化能力?我们首次提出FS-VFM,一种可扩展的自监督预训练框架,用于学习真实人脸图像的基础表征。引入三种学习目标(3C),协同掩码图像建模(MIM)与实例判别(ID),使模型能够编码人脸的局部模式与全局语义。具体而言,我们设计了多种人脸掩码策略,其中CRFR-P通过显式引导模型关注区域内部一致性与区域间连贯性。同时提出可靠自蒸馏机制,无缝耦合MIM与ID以建立局部到全局的对应关系。预训练后,普通视觉变压器(ViTs)可作为通用视觉基础模型,应用于跨数据集深度伪造检测、跨域人脸反欺骗及未见扩散人脸伪造检测。为高效迁移预训练的FS-VFM,进一步提出轻量级的FS-Adapter,其基于冻结主干网络,采用新型真实锚点对比目标。在11个公共基准上的大量实验表明,我们的FS-VFM始终优于多种视觉基础模型,涵盖自然与人脸领域、全监督、弱监督与自监督范式、小、中、大尺寸的ViT规模,甚至超越当前最优任务特定方法;而FS-Adapter则提供优异的效率-性能权衡。代码与模型详见https://fsfm-3c.github.io/fsvfm.html。
原文摘要 · Abstract (English)
With abundant, unlabeled real faces, how can we learn robust and transferable facial representations to boost generalization across various face security tasks? We make the first attempt and propose FS-VFM, a scalable self-supervised pre-training framework, to learn fundamental representations of real face images. We introduce three learning objectives, namely 3C, that synergize masked image modeling (MIM) and instance discrimination (ID), empowering FS-VFM to encode both local patterns and global semantics of real faces. Specifically, we formulate various facial masking strategies for MIM and devise a simple yet effective CRFR-P masking, which explicitly prompts the model to pursue meaningful intra-region Consistency and challenging inter-region Coherency. We present a reliable self-distillation mechanism that seamlessly couples MIM with ID to establish underlying local-to-global Correspondence. After pre-training, vanilla vision transformers (ViTs) serve as universal Vision Foundation Models for downstream Face Security tasks: cross-dataset deepfake detection, cross-domain face anti-spoofing, and unseen diffusion facial forensics. To efficiently transfer the pre-trained FS-VFM, we further propose FS-Adapter, a lightweight plug-and-play bottleneck atop the frozen backbone with a novel real-anchor contrastive objective. Extensive experiments on 11 public benchmarks demonstrate that our FS-VFM consistently generalizes better than diverse VFMs, spanning natural and facial domains, fully, weakly, and self-supervised paradigms, small, base, and large ViT scales, and even outperforms SOTA task-specific methods, while FS-Adapter offers an excellent efficiency-performance trade-off. The code and models are available on https://fsfm-3c.github.io/fsvfm.html.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。