为名人面孔定制防深度伪造系统,能精准识别并解释伪造痕迹。
Guard Me If You Know Me: Protecting Specific Face-Identity from Deepfakes

- 用多模态大模型学习特定人脸的精细特征
- 在7种换脸和7种合成技术下检测准确率超90%
- 适合需要可解释性防护的公众人物或高危身份
在数字时代,保护个人身份免受深度伪造攻击日益重要,尤其对名人和政界人士而言,其面部数据易获取且常被滥用。现有大多数深度伪造检测方法聚焦通用场景,忽视了已知身份(如‘VIP’个体)的先验信息——即真实面部数据。本文提出 extbf{VIPGuard},一种统一的多模态框架,旨在捕捉特定身份的细粒度、全维度面部表征,对比潜在伪造或相似面孔,并基于比较结果进行准确且可解释的判断。框架包含三个阶段:首先微调多模态大语言模型(MLLM),学习详细结构化面部属性;其次进行身份级判别学习,区分高度相似面孔间的细微差异(包括真实与伪造变体);最后引入用户定制化机制,建模目标身份的独特特征,并通过MLLM进行语义推理,实现个性化、可解释的检测。相比传统依赖低层视觉线索且无解释性的检测器,以及缺乏对具体身份深入理解的其他MLLM模型,本方法显著提升性能。为评估效果,我们构建了名为 extbf{VIPBench} 的身份感知基准,涵盖最新的7种人脸换脸和7种完整人脸生成技术。代码已开源:https://github.com/KQL11/VIPGuard。
原文摘要 · Abstract (English)
Securing personal identity against deepfake attacks is increasingly critical in the digital age, especially for celebrities and political figures whose faces are easily accessible and frequently targeted. Most existing deepfake detection methods focus on general-purpose scenarios and often ignore the valuable prior knowledge of known facial identities, e.g., "VIP individuals" whose authentic facial data are already available. In this paper, we propose \textbf{VIPGuard}, a unified multimodal framework designed to capture fine-grained and comprehensive facial representations of a given identity, compare them against potentially fake or similar-looking faces, and reason over these comparisons to make accurate and explainable predictions. Specifically, our framework consists of three main stages. First, fine-tune a multimodal large language model (MLLM) to learn detailed and structural facial attributes. Second, we perform identity-level discriminative learning to enable the model to distinguish subtle differences between highly similar faces, including real and fake variations. Finally, we introduce user-specific customization, where we model the unique characteristics of the target face identity and perform semantic reasoning via MLLM to enable personalized and explainable deepfake detection. Our framework shows clear advantages over previous detection works, where traditional detectors mainly rely on low-level visual cues and provide no human-understandable explanations, while other MLLM-based models often lack a detailed understanding of specific face identities. To facilitate the evaluation of our method, we built a comprehensive identity-aware benchmark called \textbf{VIPBench} for personalized deepfake detection, involving the latest 7 face-swapping and 7 entire face synthesis techniques for generation. The code is available at https://github.com/KQL11/VIPGuard .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。