arXiv:2604.09101cs.CRcs.AI2026-04中稿 · CVPR

通过逆向生成异常触发器,检测提示调优版CLIP模型是否存在后门。

CLIP-Inspector: Model-Level Backdoor Detection for Prompt-Tuned CLIP via OOD Trigger Inversion

  • 基于白盒访问和无标签分布外图像,逆向推导每类可能的触发器。
  • 仅用1000张分布外图像,单轮训练即可重建有效触发器,检测准确率达94%。
  • 可修复后门模型,适合外包训练场景下模型安全性验证与修复。

资源有限的组织越来越多地将模型训练外包给机器学习即服务(MLaaS)提供商,后者通过提示调优而非从头训练来适配视觉语言模型(如CLIP)至下游任务。这种半诚实环境下,恶意提供商可遵循提示调优流程植入后门,使特定触发输入被错误分类为攻击者指定类别,甚至对分布外(OOD)数据也生效。此类后门不破坏编码器,现有针对编码器污染的检测方法无效;其他数据级净化方法也无法回答‘交付模型是否被植入后门’这一核心问题。为此,我们提出CLIP-Inspector(CI),一种专为提示调优的CLIP模型设计的模型级后门检测方法。假设具备白盒访问权限及一批无标签的OOD图像,CI可重构每个类别的潜在触发器,判断模型是否具有后门行为。此外,我们证明使用CI重构的触发器对正确标注的触发样本进行微调,能有效重置模型、降低后门效能。在十个数据集和四种后门攻击上的实验表明,CI仅需1000张OOD图像,在单个训练周期内即可重建有效触发器,检测准确率达94%(50个模型中47个被正确识别)。相比改进的触发器逆向基线,CI的AUROC得分显著更高(0.973 vs 0.495/0.687),从而实现提示调优模型的安全验证与事后修复。

原文摘要 · Abstract (English)

Organisations with limited data and computational resources increasingly outsource model training to Machine Learning as a Service (MLaaS) providers, who adapt vision-language models (VLMs) such as CLIP to downstream tasks via prompt tuning rather than training from scratch. This semi-honest setting creates a security risk where a malicious provider can follow the prompt-tuning protocol yet implant a backdoor, forcing triggered inputs to be classified into an attacker-chosen class, even for out-of-distribution (OOD) data. Such backdoors leave encoders untouched, making them undetectable to existing methods that focus on encoder corruption. Other data-level methods that sanitize data before training or during inference, also fail to answer the critical question, "Is the delivered model backdoored or not?" To address this model-level verification problem, we introduce CLIP-Inspector (CI), a backdoor detection method designed for prompt-tuned CLIP models. Assuming white-box access to the delivered model and a pool of unlabeled OOD images, CI reconstructs possible triggers for each class to determine if the model exhibits backdoor behaviour or not. Additionally, we demonstrate that using CI's reconstructed trigger for fine-tuning on correctly labeled triggered inputs enables us to re-align the model and reduce backdoor effectiveness. Through extensive experiments across ten datasets and four backdoor attacks, we demonstrate that CI can reconstruct effective triggers in a single epoch using only 1,000 OOD images, achieving a 94% detection accuracy (47/50 models). Compared to adapted trigger-inversion baselines, CI yields a markedly higher AUROC score (0.973 vs 0.495/0.687), thus enabling the vetting and post-hoc repair of prompt-tuned CLIP models to ensure safe deployment.

后门检测CLIP提示调优模型安全

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。