用双流结构融合视觉语言模型,提升深伪检测泛化能力
Deepfake Forensics Adapter: A Dual-Stream Network for Generalizable Deepfake Detection
- 双流设计:全局特征适配+局部异常感知,结合先验知识
- 在DFDC数据集上达0.836视频级AUC,比之前方法提升4.8%
- 无需修改预训练模型参数,适合应对新型伪造手法
深度伪造生成技术的快速发展对公共安全构成严重威胁,现有检测方法在泛化到新型伪造模式方面存在局限。本文提出深度伪造取证适配器(DFA),一种基于视觉-语言基础模型的双流框架,通过不修改CLIP参数即可实现专业化深伪检测。该方法包含三个核心组件:全局特征适配器用于捕捉图像内容中的全局不一致;局部异常流利用人脸结构先验增强局部伪造线索感知;交互式融合分类器采用Transformer编码器促进全局与局部特征深度交互。在帧级和视频级基准测试中,DFA展现出卓越的泛化能力,尤其在具有挑战性的DFDC数据集上,帧级AUC/EER达到0.816/0.256,视频级达0.836/0.251,视频级AUC相比前序方法提升4.8%。该框架不仅性能领先,也为构建具备强泛化能力的鲁棒深伪检测系统提供了可行路径。
原文摘要 · Abstract (English)
The rapid advancement of deepfake generation techniques poses significant threats to public safety and causes societal harm through the creation of highly realistic synthetic facial media. While existing detection methods demonstrate limitations in generalizing to emerging forgery patterns, this paper presents Deepfake Forensics Adapter (DFA), a novel dual-stream framework that synergizes vision-language foundation models with targeted forensics analysis. Our approach integrates a pre-trained CLIP model with three core components to achieve specialized deepfake detection by leveraging the powerful general capabilities of CLIP without changing CLIP parameters: 1) A Global Feature Adapter is used to identify global inconsistencies in image content that may indicate forgery, 2) A Local Anomaly Stream enhances the model's ability to perceive local facial forgery cues by explicitly leveraging facial structure priors, and 3) An Interactive Fusion Classifier promotes deep interaction and fusion between global and local features using a transformer encoder. Extensive evaluations of frame-level and video-level benchmarks demonstrate the superior generalization capabilities of DFA, particularly achieving state-of-the-art performance in the challenging DFDC dataset with frame-level AUC/EER of 0.816/0.256 and video-level AUC/EER of 0.836/0.251, representing a 4.8% video AUC improvement over previous methods. Our framework not only demonstrates state-of-the-art performance, but also points out a feasible and effective direction for developing a robust deepfake detection system with enhanced generalization capabilities against the evolving deepfake threats. Our code is available at https://github.com/Liao330/DFA.git
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。