用三元组学习提升人脸伪造检测泛化能力,能识别未见过的伪造手法。
Trident: Detecting Face Forgeries with Adversarial Triplet Learning
- 基于孪生网络的三元组学习,捕捉细微伪造特征。
- 在多个数据集上优于现有方法,对未知伪造仍保持高准确率。
- 适合需要鲁棒检测能力的研究者和安全应用开发者。
随着深度神经网络生成的人脸伪造技术日益精进,数字媒体中的人脸篡改检测面临严峻挑战,维护数字内容真实性与防范视觉误导至关重要。当前主流检测模型依赖特定领域数据的监督训练,难以应对未见过的伪造方法。为此,我们提出Trident框架,采用孪生网络结构的三元组学习,增强对多种伪造手段的适应性。Trident通过精心构建的三元组训练,分离出伪造样本间的细微差异,捕获区分真实与伪造图像的细粒度特征,同时控制其他变量影响。为进一步提升泛化能力,引入伪造鉴别器进行域对抗训练,引导嵌入模型生成与伪造无关的表示,提高对未知篡改的鲁棒性。此外,通过阻断分类头到嵌入模型的梯度传播,避免因特定伪造伪影导致的过拟合。在多个基准测试上的全面评估及消融实验验证了该框架的有效性。代码将开源于GitHub。
原文摘要 · Abstract (English)
As face forgeries generated by deep neural networks become increasingly sophisticated, detecting face manipulations in digital media has posed a significant challenge, underscoring the importance of maintaining digital media integrity and combating visual disinformation. Current detection models, predominantly based on supervised training with domain-specific data, often falter against forgeries generated by unencountered techniques. In response to this challenge, we introduce \textit{Trident}, a face forgery detection framework that employs triplet learning with a Siamese network architecture for enhanced adaptability across diverse forgery methods. \textit{Trident} is trained on curated triplets to isolate nuanced differences of forgeries, capturing fine-grained features that distinguish pristine samples from manipulated ones while controlling for other variables. To further enhance generalizability, we incorporate domain-adversarial training with a forgery discriminator. This adversarial component guides our embedding model towards forgery-agnostic representations, improving its robustness to unseen manipulations. In addition, we prevent gradient flow from the classifier head to the embedding model, avoiding overfitting induced by artifacts peculiar to certain forgeries. Comprehensive evaluations across multiple benchmarks and ablation studies demonstrate the effectiveness of our framework. We will release our code in a GitHub repository.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。