混合EfficientNet与Swin Transformer,高效识别生成人脸伪造。
Swin Meets EfficientNet: Lightweight Architectures for GAN-Based Face Forensics

- 用EfficientNet提取局部特征,Swin Transformer捕捉全局结构。
- 在5000张测试图上达99%准确率与99.44%召回率。
- 轻量级设计适合实际部署,尤其适合资源受限场景。
现代生成模型(如GAN、扩散模型和自回归系统)已能生成几乎无法与真实照片区分的人脸图像,加剧了身份盗用、欺诈和虚假信息传播的风险。本文聚焦于基于GAN的合成人脸检测,仅通过图像分析实现高效识别。现有系统多依赖CNN或全局视觉变换器(ViT),前者擅长纹理特征但缺乏上下文理解,后者虽能捕捉长程结构却计算开销大。本研究探索三种Swin-Transformer架构:从零训练的紧凑Swin、ImageNet-1K预训练的Swin-Tiny与Swin-Small,以及创新的EfficientNet-B0+Swin混合模型。在包含14万张真实与伪造人脸的140K Real and Fake Faces数据集上评估,该混合模型在5,000张测试图像上达到99%准确率与99.44%召回率,优于纯Swin变体及先前的CNN基线。结果表明,结合层次化CNN特征与移窗自注意力机制,是一种高效且轻量的GAN伪造人脸检测方法。
原文摘要 · Abstract (English)
Modern generative models, such as GANs, diffusion architectures, and autoregressive systems, now produce facial images that are nearly indistinguishable from authentic photographs. This capability makes detecting forged images increasingly difficult, raising serious concerns about identity theft, fraud, and misinformation campaigns. Our research focuses specifically on GAN-generated synthetic faces, which underpin many face-centric deepfakes, and investigates efficient detection approaches using image analysis alone. Existing detection systems rely heavily on either convolutional neural networks (CNNs) or global vision transformers. While CNNs excel at identifying texture-based local features, they struggle with broader contextual understanding. Traditional Vision Transformer (ViT) models can capture long-range structures effectively, but demand substantial computational resources. Our work explores Swin-Transformer-based architectures across three implementations: a compact Swin Transformer trained from the ground up, ImageNet-1K pre-trained Swin-Tiny and Swin-Small models adapted for binary classification, and a novel hybrid combining EfficientNet-B0's convolutional processing with a Swin Transformer backend. We evaluated all models using the 140K Real and Fake Faces dataset, which includes StyleGAN-generated fake faces alongside authentic images from Flickr and DFDC, with balanced splits for training, validation, and testing. The EfficientNetB0+Swin hybrid achieved 99% accuracy and a 99.44% recall on 5,000 test images, outperforming both pure Swin variants and a previous CNN-only baseline on this dataset. Our results suggest that combining hierarchical CNN features with shifted-window self-attention provides an efficient and computationally lightweight method for detecting GAN-generated synthetic faces.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。