解决伪造图像检测中训练冲突问题,提升模型泛化能力。
Suppressing Gradient Conflict for Generalizable Deepfake Detection

- 通过搜索替代更新向量缓解真实与合成伪造图像的梯度冲突。
- 在多个基准上实现领先检测精度与跨域泛化性能。
- 适合需要高泛化能力的深度伪造检测场景。
鲁棒的深度伪造检测模型需具备对训练数据之外不断演化的伪造技术的泛化能力。一种有前景的方法是使用在线生成的、包含广泛通用特征的伪造图像扩充训练数据。然而,在深度伪造检测中,同时训练原始数据和在线生成的伪造图像反而可能导致性能下降,这违背了‘更多源域数据应提升准确率’的普遍认知。通过实证分析,我们发现性能下降源于反向传播中的梯度冲突,迫使模型在源域准确率与目标域泛化之间权衡。为此,我们提出冲突抑制型深度伪造检测(CS-DFD)框架,通过两个协同模块缓解梯度冲突:第一,更新向量搜索(UVS)模块在初始梯度附近寻找替代更新向量,以调和真实与合成伪造图像之间的差异;进一步将搜索转化为极值优化问题,得到唯一最优更新向量,能同时最小化两类数据的损失。第二,冲突梯度降低(CGR)模块通过新型冲突下降损失,强制学习低冲突的特征嵌入空间,惩罚梯度方向不一致,引导具有对齐、非冲突梯度的表示学习。UVS与CGR的协同作用在参数优化与表示学习中均减轻了梯度干扰。在多个深度伪造基准上的实验表明,CS-DFD在域内检测准确率与跨域泛化上均达到当前最优性能。
原文摘要 · Abstract (English)
Robust deepfake detection models must be capable of generalizing to ever-evolving manipulation techniques beyond training data. A promising strategy is to augment the training data with online synthesized fake images containing broadly generalizable artifacts. However, in the context of deepfake detection, it is surprising that jointly training on both original and online synthesized forgeries may result in degraded performance. This contradicts the common belief that incorporating more source-domain data should enhance detection accuracy. Through empirical analysis, we trace this degradation to gradient conflicts during backpropagation which force a trade-off between source domain accuracy and target domain generalization. To overcome this issue, we propose a Conflict-Suppressed Deepfake Detection (CS-DFD) framework that explicitly mitigates the gradient conflict via two synergistic modules. First, an Update Vector Search (UVS) module searches for an alternative update vector near the initial gradient vector to reconcile the disparities of the original and online synthesized forgeries. By further transforming the search process into an extremum optimization problem, UVS yields the uniquely update vector, which maximizes the simultaneous loss reductions for each data type. Second, a Conflict Gradient Reduction (CGR) module enforces a low-conflict feature embedding space through a novel Conflict Descent Loss. This loss penalizes misaligned gradient directions and guides the learning of representations with aligned, non-conflicting gradients. The synergy of UVS and CGR alleviates gradient interference in both parameter optimization and representation learning. Experiments on multiple deepfake benchmarks demonstrate that CS-DFD achieves state-of-the-art performance in both in-domain detection accuracy and cross-domain generalization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。