通过彩票理论找出识别深度伪造的关键特征,实现高效轻量检测。
Uncovering Critical Features for Deepfake Detection through the Lottery Ticket Hypothesis
- 用彩票理论迭代剪枝,找到维持性能的微型核心网络。
- 在OpenForensic上80%稀疏度下仍保持56.2%准确率,仅3000参数。
- 剪枝后模型聚焦关键面部区域,且跨数据集可迁移。
深度伪造技术的进步带来了越来越逼真的合成媒体,严重威胁信息真实性和社会信任。尽管现有检测方法表现良好,但其内在机制仍不清晰,且模型庞大难以在资源受限环境部署。本研究探索彩票理论(LTH)在深度伪造检测中的应用,旨在识别关键判别特征。通过在OpenForensic和FaceForensics++数据集上对MesoNet、CNN-5和ResNet-18进行大量实验,发现检测网络中存在“中奖彩票”——即在高稀疏度下仍能保持性能的子网络。结果显示,MesoNet在OpenForensic上达到80%稀疏度时,准确率为56.2%,参数仅3,000个,约为基线准确率62.6%的90%。所提出的基于LTH的迭代幅度剪枝方法始终优于一次性剪枝。通过Grad-CAM可视化,发现剪枝后网络仍聚焦于关键面部区域。此外,验证了中奖彩票在不同数据集间的可迁移性,表明其在高效可部署检测系统中的潜力。
原文摘要 · Abstract (English)
Recent advances in deepfake technology have created increasingly convincing synthetic media that poses significant challenges to information integrity and social trust. While current detection methods show promise, their underlying mechanisms remain poorly understood, and the large sizes of their models make them challenging to deploy in resource-limited environments. This study investigates the application of the Lottery Ticket Hypothesis (LTH) to deepfake detection, aiming to identify the key features crucial for recognizing deepfakes. We examine how neural networks can be efficiently pruned while maintaining high detection accuracy. Through extensive experiments with MesoNet, CNN-5, and ResNet-18 architectures on the OpenForensic and FaceForensics++ datasets, we find that deepfake detection networks contain winning tickets, i.e., subnetworks, that preserve performance even at substantial sparsity levels. Our results indicate that MesoNet retains 56.2% accuracy at 80% sparsity on the OpenForensic dataset, with only 3,000 parameters, which is about 90% of its baseline accuracy (62.6%). The results also show that our proposed LTH-based iterative magnitude pruning approach consistently outperforms one-shot pruning methods. Using Grad-CAM visualization, we analyze how pruned networks maintain their focus on critical facial regions for deepfake detection. Additionally, we demonstrate the transferability of winning tickets across datasets, suggesting potential for efficient, deployable deepfake detection systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。