通过分层掩码与多伪造子空间分解,提升深度伪造检测的跨数据集泛化能力。
Generalizable Deepfake Detection Based on Forgery-aware Layer Masking and Multi-artifact Subspace Decomposition
- 按梯度特性筛选敏感层,仅更新关键部分以保护预训练语义
- 用SVD分解权重,分离出多个可学习的伪造特征子空间
- 通过正交性约束减少冗余,适合真实场景下复杂伪造检测
深度伪造检测在跨数据集和复杂真实场景中仍具挑战性,主要因伪造痕迹模式在不同伪造方法间差异显著。现有方法常依赖全参数微调或辅助监督,易过度关注特定伪造线索而破坏预训练表征,削弱泛化能力。为此,提出FMSD框架,基于伪造感知层掩码与多伪造子空间分解。首先,通过分析层级梯度的偏差-方差特性,识别伪造敏感层,选择性更新以减少对预训练表示的干扰。在此基础上,利用奇异值分解(SVD)将选定层权重分解为语义子空间和多个可学习的伪造子空间,分别建模异质且互补的伪造模式,同时保留预训练语义信息。进一步引入正交性和谱一致性约束,规范伪造子空间,降低其内部冗余并保持预训练权重的整体谱结构。
原文摘要 · Abstract (English)
Deepfake detection remains highly challenging, particularly in cross-dataset scenarios and complex real-world settings. This challenge mainly arises because artifact patterns vary substantially across different forgery methods, whereas adapting pretrained models to such artifacts often overemphasizes forgery-specific cues and disturbs semantic representations, thereby weakening generalization. Existing approaches typically rely on full-parameter fine-tuning or auxiliary supervision to improve discrimination. However, they often struggle to model diverse forgery artifacts without compromising pretrained representations. To address these limitations, we propose FMSD, a deepfake detection framework built upon Forgery-aware Layer Masking and Multi-Artifact Subspace Decomposition. Specifically, Forgery-aware Layer Masking evaluates the bias-variance characteristics of layer-wise gradients to identify forgery-sensitive layers, thereby selectively updating them while reducing unnecessary disturbance to pretrained representations. Building upon this, Multi-Artifact Subspace Decomposition further decomposes the selected layer weights via Singular Value Decomposition (SVD) into a semantic subspace and multiple learnable artifact subspaces. These subspaces are optimized to capture heterogeneous and complementary forgery artifacts, enabling effective modeling of diverse forgery patterns while preserving pretrained semantic representations. Furthermore, orthogonality and spectral consistency constraints are imposed to regularize the artifact subspaces, reducing redundancy across them while preserving the overall spectral structure of pretrained weights.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。