新方法让检测模型随伪造技术演进持续学习,避免遗忘旧类型。
DevFD: Developmental Face Forgery Detection by Learning Shared and Orthogonal LoRA Subspaces
- 用分组LoRA专家架构区分真实与伪造人脸学习路径
- 通过正交方向防止新旧伪造类型间知识干扰
- 适合需持续更新的实时伪造检测场景
逼真的数字人脸生成与篡改技术迅猛发展,带来严重社会风险。现有检测模型难以跟上快速多变的生成技术迭代。为应对不断涌现的新伪造类型,需在计算和数据受限条件下快速适应新域,同时不遗忘旧知识。本文将人脸伪造检测视为持续学习问题,提出开发式混合专家(MoE)架构,以LoRA作为独立专家。专家分为两组:Real-LoRA用于学习并优化真实人脸特征,多个Fake-LoRAs分别捕获不同伪造类型增量信息。为防止灾难性遗忘,强制Fake-LoRAs学习方向与已建立子空间正交,并在训练中引入正交梯度,消除各任务间的梯度干扰。在增量数据集与增量伪造类型两种测试协议下,实验验证了该方法的有效性。
原文摘要 · Abstract (English)
The rise of realistic digital face generation and manipulation poses significant social risks. The primary challenge lies in the rapid and diverse evolution of generation techniques, which often outstrip the detection capabilities of existing models. To defend against the ever-evolving new types of forgery, we need to enable our model to quickly adapt to new domains with limited computation and data while avoiding forgetting previously learned forgery types. In this work, we posit that genuine facial samples are abundant and relatively stable in acquisition methods, while forgery faces continuously evolve with the iteration of manipulation techniques. Given the practical infeasibility of exhaustively collecting all forgery variants, we frame face forgery detection as a continual learning problem and allow the model to develop as new forgery types emerge. Specifically, we employ a Developmental Mixture of Experts (MoE) architecture that uses LoRA models as its individual experts. These experts are organized into two groups: a Real-LoRA to learn and refine knowledge of real faces, and multiple Fake-LoRAs to capture incremental information from different forgery types. To prevent catastrophic forgetting, we ensure that the learning direction of Fake-LoRAs is orthogonal to the established subspace. Moreover, we integrate orthogonal gradients into the orthogonal loss of Fake-LoRAs, preventing gradient interference throughout the training process of each task. Experimental results under both the datasets and manipulation types incremental protocols demonstrate the effectiveness of our method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。