通过跨模态矛盾检测,提升深度伪造视频识别精度与鲁棒性
CAM-VFD: Cross-Attention Multimodal Video Forgery Detection

- 利用交叉注意力融合视觉、运动与深度信息,捕捉跨模态不一致
- 在GenVidBench上达到95.31%准确率,多指标优于现有方法
- 对压缩、噪声等干扰保持稳定,适合真实场景应用
深度伪造技术快速发展,严重威胁多媒体取证与信息真实性。现有检测器依赖单模态信号,无法发现生成模型在同模态内保持一致但跨模态存在矛盾的特征。本文提出CAM-VFD框架,将跨模态矛盾作为定向取证信号,采用基于CLIP的外观表示作为查询,对齐VideoMAE运动特征与MiDaS深度特征,实现视觉、时间与几何证据间的交叉注意力融合。通过跨模态注意力差异分析,真实与伪造样本分布具有统计显著差异(p<0.001,Cohen's d=0.68)。在两个生成视频基准测试中表现稳定:GenVidBench Top-1准确率达95.31%;GenVideo上准确率93.43%、F1分数90.63%、AUROC达96.56%。该模型在压缩、噪声、模糊及对抗扰动下仍具稳健性,表明跨模态推理可增强媒体取证鲁棒性。代码已公开于https://github.com/Hoda-Osama/CAM-VFD/tree/main。
原文摘要 · Abstract (English)
The rapid advancement of Deepfake technologies and video manipulation tools poses a critical challenge to multimedia forensics, judicial evidence integrity, and information authenticity. Current detectors rely on single-modality signals, treating appearance, geometry, and motion independently. However, advanced generators maintain within-modality consistency while producing cross-modal contradictions, which are forensically discriminative but invisible to any single-modal detector. We propose CAM-VFD, a Cross-Attention Multimodal Video Forgery Detection framework that models cross-modal contradiction as a directional forensic signal. The framework uses a cross-attention fusion mechanism in which CLIP-based appearance representations serve as queries against VideoMAE motion features and MiDaS depth features, enabling the identification of contradictions between visual, temporal, and geometric evidence. We examine this design through cross-modal attention discrepancy analysis, observing statistically separable real and fake distributions ($p<0.001$, Cohen's $d=0.68$). Experimental results on two generative video benchmarks indicate consistent performance, with 95.31\% Top-1 accuracy on GenVidBench and 93.43\% accuracy, 90.63\% F1-score, and 96.56\% AUROC on GenVideo. Moreover, CAM-VFD demonstrates stable performance under compression, noise, blur, and adversarial perturbations, suggesting that cross-modal reasoning may improve robustness in media forensics. The code is publicly available at \url{https://github.com/Hoda-Osama/CAM-VFD/tree/main}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。