利用视觉语言语义提升伪造视频检测能力,效果优于现有方法。
Unleashing Vision-Language Semantics for Deepfake Video Detection
- 通过独立学习器捕捉细微伪造痕迹,保留预训练对齐知识。
- 提出身份感知的跨模态评分,显著提升检测精度。
- 适合关注伪造视频检测与多模态模型应用的研究者。
近期深度伪造视频检测研究显示,CLIP等预训练视觉语言模型在跨身份检测中具备强大泛化能力。然而,现有方法仅依赖视觉特征,忽视其最核心优势——潜空间中的丰富视觉语言语义。本文提出VLAForge框架,充分释放跨模态语义潜力以增强检测判别力。该工作i)通过ForgePerceiver增强视觉感知,独立学习细粒度与整体伪造线索,同时保持预训练的视觉语言对齐(VLA)知识;ii)引入身份感知的VLA评分,结合跨模态语义与ForgePerceiver所学伪造线索。该评分通过融合身份先验提示词,捕获针对个体的身份真实性线索,实现更优的判别性跨模态语义。在多个视频伪造检测基准上(包括经典换脸伪造与最新全脸生成伪造),VLAForge在帧级与视频级均显著超越当前最优方法。代码已开源:https://github.com/mala-lab/VLAForge。
原文摘要 · Abstract (English)
Recent Deepfake Video Detection (DFD) studies have demonstrated that pre-trained Vision-Language Models (VLMs) such as CLIP exhibit strong generalization capabilities in detecting artifacts across different identities. However, existing approaches focus on leveraging visual features only, overlooking their most distinctive strength -- the rich vision-language semantics embedded in the latent space. We propose VLAForge, a novel DFD framework that unleashes the potential of such cross-modal semantics to enhance model's discriminability in deepfake detection. This work i) enhances the visual perception of VLM through a ForgePerceiver, which acts as an independent learner to capture diverse, subtle forgery cues both granularly and holistically, while preserving the pretrained Vision-Language Alignment (VLA) knowledge, and ii) provides a complementary discriminative cue -- Identity-Aware VLA score, derived by coupling cross-modal semantics with the forgery cues learned by ForgePerceiver. Notably, the VLA score is augmented by an identity prior-informed text prompting to capture authenticity cues tailored to each identity, thereby enabling more discriminative cross-modal semantics. Comprehensive experiments on video DFD benchmarks, including classical face-swapping forgeries and recent full-face generation forgeries, demonstrate that our VLAForge substantially outperforms state-of-the-art methods at both frame and video levels. Code is available at https://github.com/mala-lab/VLAForge.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。