用低秩适配补丁绕过深度伪造防御，暴露其脆弱性
LoRA Patching: Exposing the Fragility of Proactive Defenses against Deepfakes
- 通过可插拔的低秩适配补丁，动态控制攻击效果
- 仅用1000张人脸、一次训练就攻破多个主流防御
- 既可攻击也可防御，能嵌入可见警示信息
深度伪造带来严重社会风险，促使研究者开发主动防御机制，即在人脸图像中嵌入对抗扰动以阻止篡改。然而本文揭示，此类预置防御普遍存在鲁棒性和可靠性不足的问题。我们提出一种新方法——低秩适配（LoRA）补丁，将可学习的补丁注入深度伪造生成器，从而绕过当前最先进的防御体系。该方法引入可学习门控机制，动态调控补丁效果并防止微调时梯度爆炸。同时设计多模态特征对齐（MMFA）损失，使对抗输出的语义特征与目标输出保持一致。此外，我们还提出防御型LoRA补丁，可在生成结果中嵌入可见警告，作为应对新发现漏洞的补充方案。仅需1000个面部样本和单次训练周期，该方法即可成功突破多种主动防御。结果揭示了现有防御范式的重大缺陷，凸显构建更鲁棒防御策略的紧迫性。代码已开源：https://github.com/ZOMIN28/LoRA-Patching。
原文摘要 · Abstract (English)
Deepfakes pose significant societal risks, motivating the development of proactive defenses that embed adversarial perturbations in facial images to prevent manipulation. However, in this paper, we show that these preemptive defenses often lack robustness and reliability. We propose a novel approach, Low-Rank Adaptation (LoRA) patching, which injects a plug-and-play LoRA patch into Deepfake generators to bypass state-of-the-art defenses. A learnable gating mechanism adaptively controls the effect of the LoRA patch and prevents gradient explosions during fine-tuning. We also introduce a Multi-Modal Feature Alignment (MMFA) loss, encouraging the features of adversarial outputs to align with those of the desired outputs at the semantic level. Beyond bypassing, we present defensive LoRA patching, embedding visible warnings in the outputs as a complementary solution to mitigate this newly identified security vulnerability. With only 1,000 facial examples and a single epoch of fine-tuning, LoRA patching successfully defeats multiple proactive defenses. These results reveal a critical weakness in current paradigms and underscore the need for more robust Deepfake defense strategies. Our code is available at https://github.com/ZOMIN28/LoRA-Patching.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。