对比了真假视频归属模型在跨数据集上的泛化能力,发现二分类模型更稳定。
Do DeepFake Attribution Models Generalize?
- 用五种主流模型测试六大数据集,比较二分类与多分类归属模型表现
- 二分类模型跨数据集准确率更高,但多分类模型能识别具体伪造方法
- 高质量数据和对比学习可提升归属模型性能,适合需要溯源的场景
近期深度伪造生成技术的进步及开源工具的普及,大幅降低了合成媒体的制作门槛,严重威胁在线信息的真实性和公信力。现有深度伪造检测研究多聚焦于二分类模型,但这类模型将所有篡改技术视为等同,忽视了不同方法引入的独特伪影与视觉线索。少数研究探索了深度伪造归属模型,这类模型在实际应用中至关重要,因其可揭示具体伪造手段,提升用户对结果的信任与可解释性。本文采用五种先进骨干模型,在六个深度伪造数据集上开展系统实验:首先比较二分类与多分类模型在跨数据集上的泛化能力;其次评估归属模型在未知数据集上识别已见伪造方法的准确性,揭示相同伪造方法下的数据分布偏移;最后分析对比学习对跨数据集泛化性能的提升效果。结果表明,尽管二分类模型具有更强的泛化能力,但更大模型、对比学习方法以及更高数据质量可显著提升归属模型性能。代码已公开于GitHub。
原文摘要 · Abstract (English)
Recent advancements in DeepFake generation, along with the proliferation of open-source tools, have significantly lowered the barrier for creating synthetic media. This trend poses a serious threat to the integrity and authenticity of online information, undermining public trust in institutions and media. State-of-the-art research on DeepFake detection has primarily focused on binary detection models. A key limitation of these models is that they treat all manipulation techniques as equivalent, despite the fact that different methods introduce distinct artifacts and visual cues. Only a limited number of studies explore DeepFake attribution models, although such models are crucial in practical settings. By providing the specific manipulation method employed, these models could enhance both the perceived trustworthiness and explainability for end users. In this work, we leverage five state-of-the-art backbone models and conduct extensive experiments across six DeepFake datasets. First, we compare binary and multi-class models in terms of cross-dataset generalization. Second, we examine the accuracy of attribution models in detecting seen manipulation methods in unknown datasets, hence uncovering data distribution shifts on the same DeepFake manipulations. Last, we assess the effectiveness of contrastive methods in improving cross-dataset generalization performance. Our findings indicate that while binary models demonstrate better generalization abilities, larger models, contrastive methods, and higher data quality can lead to performance improvements in attribution models. The code of this work is available on GitHub.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。