用文本提示分离伪造特征，提升人脸伪造检测泛化能力
Generalizable Face Forgery Detection via Separable Prompt Learning
- 通过分离图像中伪造相关与无关信息，设计双路提示学习
- 在跨数据集和跨方法测试中表现优于或媲美现有方法
- 适合需要高泛化性的伪造检测应用场景
利用CLIP进行人脸伪造检测已成为一种有前景且日益流行的研究方向。由于其通过大规模预训练获得的丰富视觉知识，现有大多数方法通常依赖CLIP的视觉编码器，而对文本模态关注有限。鉴于文本模态的指导性，我们提出可通过精心设计的文本模态指导深度伪造检测。为此，我们将重点从视觉模态转向文本模态，提出一种新的可分离提示学习策略（SePL），使CLIP能有效用于人脸伪造检测。SePL的核心思想是通过两种提示学习方式，将图像中的伪造特异性与非伪造相关信息解耦，其中前者增强检测能力。为实现该解耦，我们设计了跨模态对齐策略及专用目标函数。大量实验表明，通过此简单适配，本方法在跨数据集和跨方法评估下均取得具有竞争力甚至更优的性能，凸显其强大的泛化能力。代码已公开于 https://github.com/OUC-YER/SePL-DeepfakeDetection。
原文摘要 · Abstract (English)
Detecting face forgeries using CLIP has recently emerged as a promising and increasingly popular research direction. Owing to its rich visual knowledge acquired through large-scale pretraining, most existing methods typically rely on the visual encoder of CLIP, while paying limited attention to the text modality. Given the instructive nature of the text modality, we posit that it can be leveraged to instruct Deepfake detection with meticulous design. Accordingly, we shift the focus from the visual modality to the text modality and propose a new Separable Prompt Learning strategy (SePL) that enables CLIP to serve as an effective face forgery detector. The core idea of SePL is to disentangle forgery-specific and forgery-irrelevant information in images via two types of prompt learning, with the former enhancing detection. To achieve this disentangle, we describe a cross-modality alignment strategy and a set of dedicated objectives. Extensive experiments demonstrate that, with this simple adaptation, our method achieves competitive and even superior performance compared to other methods under both cross-dataset and cross-method evaluation, highlighting its strong generalizability. The codes have been released at https://github.com/OUC-YER/SePL-DeepfakeDetection
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。