提出无需标签的黑盒防御,可识别并净化自监督模型中的后门攻击。
The Platonic Defense: Backdoor Defense for Self-Supervised Encoders in the Era of Large Scale Pre-training

- 基于通用表征收敛假设,构建能量函数检测异常表征。
- 在10种以上攻击下保持性能,检测准确率提升显著。
- 适用于各类自监督模型,适合安全敏感场景使用。
自监督学习(SSL)预训练模型已成为视觉表征学习的主流,但易受后门攻击。现有防御方法在完全黑盒环境下表现不佳,因常需标签、攻击模式或训练数据。为此,我们提出一种攻击无关、模型无关、模态无关的黑盒测试时防御范式——柏拉图表征防御(Platonic Representation Defense)。该方法受柏拉图表征假说启发,认为大规模独立训练的编码器会收敛于同一底层现实的兼容投影。我们将其形式化为源表征与一组参考表征上的条件能量函数,通过噪声对比估计训练检测,通过去噪得分匹配实现表征净化。理论上,匹配与不匹配样本间的能量差有互信息下界。实验验证了该方法在多种自监督编码器和十余种攻击下的有效性,兼具表征检测与净化能力,并在多个攻击中取得显著性能提升。代码已公开。
原文摘要 · Abstract (English)
Self-supervised learning (SSL) pretrained models have become a dominant paradigm for visual representation learning, but they are vulnerable to backdoor attacks. Existing defenses struggle to defend against such attacks in a fully black-box setting because they often require access to labels, attack patterns, or training data. To tackle this issue, we propose a new attack-agnostic, model-agnostic, and modality-agnostic black-box test-time defense paradigm, called \emph{Platonic Representation Defense}. It is inspired by the Platonic Representation Hypothesis, which suggests that large-scale independently trained encoders converge toward compatible projections of the same underlying reality. We formalize this idea as a conditional energy function defined over source representations and a set of reference representations. The energy function is trained for detection through noise-contrastive estimation and for representation purification through denoising score matching. Theoretically, the energy gap between matched and mismatched samples is lower bounded by the mutual information between source and reference representations. We demonstrate the effectiveness of our method on multiple self-supervised encoders and more than 10 attacks. The method can perform both representation detection and purification, and achieves substantial performance gains across multiple attacks. Code is available \href{https://github.com/jsrdcht/Platonic-Representation-Defense}{here}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。