用参数融合方法提升深度伪造检测,无需重新训练。
Real-Aware Residual Model Merging for Deepfake Detection
- 通过低秩分解提取共性真实特征,分离生成器特异性伪影
- 融合后在跨数据集测试中准确率超联合训练基线3.2个百分点
- 新增伪造类型时只需微调单个模型即可快速更新
深度伪造生成器快速演进,导致穷尽数据收集和重复训练不切实际。我们认为模型融合天然适用于深度伪造检测:不同于标签不重叠的多任务场景,深度伪造专家共享二元判别目标,仅在生成器特异性痕迹上不同。实证表明,简单权重平均能保留真实表征,同时弱化虚假线索。基于此,我们提出无训练参数空间融合框架R²M,通过低秩因子分解估计共享真实成分,将每个专家分解为对齐真实的部分与虚假残差,用逐层秩截断去噪残差,并通过任务归一化聚合防止单一生成器主导。简洁解释表明,简单头部即可有效:真实成分在特征空间诱导出共同分离方向,截断残差仅贡献微小非轴向扰动。在同分布、跨数据集及未见数据集测试中,R²M均优于联合训练和其他融合基线。重要的是,R²M具备可组合性:当出现新伪造家族时,仅需微调一个专家并重新融合,无需完整重训练。
原文摘要 · Abstract (English)
Deepfake generators evolve quickly, making exhaustive data collection and repeated retraining impractical. We argue that model merging is a natural fit for deepfake detection: unlike generic multi-task settings with disjoint labels, deepfake specialists share the same binary decision and differ in generator-specific artifacts. Empirically, we show that simple weight averaging preserves Real representations while attenuating Fake-specific cues. Building upon these findings, we propose Real-aware Residual Model Merging (R$^2$M), a training-free parameter-space merging framework. R$^2$M estimates a shared Real component via a low-rank factorization of task vectors, decomposes each specialist into a Real-aligned part and a Fake residual, denoises residuals with layerwise rank truncation, and aggregates them with per-task norm matching to prevent any single generator from dominating. A concise rationale explains why a simple head suffices: the Real component induces a common separation direction in feature space, while truncated residuals contribute only minor off-axis variations. Across in-distribution, cross-dataset, and unseen-dataset, R$^2$M outperforms joint training and other merging baselines. Importantly, R$^2$M is also composable: when a new forgery family appears, we fine-tune one specialist and re-merge, eliminating the need for retraining.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。