提出DEFUSE框架,用生成模型检测自监督编码器的后门攻击。
DEFUSE: Generalizable Backdoor Defense for Self-Supervised Encoders with Generative Priors

- 基于生成先验的条件扩散模型,通过语义重建判断是否被植入后门。
- 在多种攻击场景下性能显著优于现有方法,且不依赖具体编码器或攻击策略。
- 适用于视觉与视觉-语言自监督模型,适合安全敏感场景使用。
自监督学习(SSL)编码器易受后门攻击,威胁视觉与视觉-语言编码器的安全性。现有防御方法通常仅针对某一类模型,且依赖难以满足的假设,如需未污染的分布内数据或预计算伪标签。为此,我们提出DEFUSE,一种通用的SSL编码器后门检测框架。受贝叶斯后验推断启发,将后门检测建模为以表示条件的图像似然估计问题,采用条件扩散生成模型进行求解。未感染的表示能生成语义一致的重构图像,而被植入后门的表示则更可能映射到攻击者目标类别或语义无关图像,导致语义偏离从而暴露后门。然而,精确似然难以计算,因高度抽象表示丢失了像素级重建所需低层信息。因此,我们放宽目标至语义重建,并在参考编码器提供的分离表示空间中评估。无需从头训练,仅微调预训练扩散模型,利用其生成先验将数据映射至自然图像流形,同时保持语义内容。大量实验表明,DEFUSE在多种攻击设置下均显著优于现有检测器,可泛化至视觉与视觉-语言编码器。尤其大幅降低对目标编码器或攻击策略先验知识的依赖。代码已开源:https://github.com/jsrdcht/DEFUSE。
原文摘要 · Abstract (English)
Self-supervised learning (SSL) encoders are vulnerable to backdoor attacks, posing threats to both visual SSL encoders and vision-language encoders. Existing defenses are typically designed for only one of these paradigms and rely on restrictive assumptions such as access to uninfected in-distribution data or precomputed pseudo-labels, which are difficult to satisfy in practice. To address these limitations, we propose DEFUSE, a generalizable backdoor detection framework for SSL encoders. Inspired by Bayesian posterior inference, we reformulate backdoor detection as a representation-conditioned image likelihood estimation problem parameterized by a conditional diffusion generative model. Uninfected representations tend to yield semantically consistent reconstructions, whereas backdoored ones are more likely to be mapped to the attacker's target class or semantically meaningless images, deviating from the original semantics and thereby exposing the backdoor. However, we find that the exact likelihood is intractable, because highly abstracted representations discard the low-level information necessary for pixel-faithful reconstruction. We therefore relax the objective to semantic reconstruction and evaluate it in a well-separated representation space provided by a reference encoder. Rather than training from scratch, we fine-tune a pretrained diffusion model, leveraging its generative prior to map data onto the natural image manifold while preserving semantic content. Extensive experiments demonstrate that DEFUSE substantially outperforms existing detectors across diverse attack settings, generalizing to both visual SSL and vision-language encoders. Notably, our method greatly reduces the reliance on prior knowledge about the victim encoder or the attack strategy. The source code is available at https://github.com/jsrdcht/DEFUSE .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。