arXiv:2601.03805cs.LG2026-01

通过对比模型差异,检测分类模型中的语义后门。

Detecting Semantic Backdoors in a Mystery Shopping Scenario

  • 用可信环境训练干净与污染模型,构建参考池。
  • 基于反向生成样本的模型距离度量可有效区分干净与污染模型。
  • 适合监管机构在神秘购物场景下验证AI服务安全性。

检测分类模型中的语义后门——即某些自然但分布外的输入可激活特定类别——是一个重要但关注较少的问题。由于缺乏明显的触发模式,这类后门比基于模式的后门更难检测。本文假设清洁训练数据集和训练流程均可知,模拟消费者保护机构进行神秘购物测试的场景。机构利用提供商资源训练模型,并检验是否存在后门。提出的方法是:在可信基础设施上训练少量干净与污染模型,构建参考模型池,并校准模型距离阈值以识别干净模型。实验分析了多种模型距离计算方法,并测试了提供商采取自适应攻击规避检测的情形。最可靠的方法是请求对抗训练;最佳模型距离测量方式是通过反向生成样本来最大化与干净样本的距离。在此设置下,方法能有效分离干净与污染模型,性能优于现有先进检测器。

原文摘要 · Abstract (English)

Detecting semantic backdoors in classification models--where some classes can be activated by certain natural, but out-of-distribution inputs--is an important problem that has received relatively little attention. Semantic backdoors are significantly harder to detect than backdoors that are based on trigger patterns due to the lack of such clearly identifiable patterns. We tackle this problem under the assumption that the clean training dataset and the training recipe of the model are both known. These assumptions are motivated by a consumer protection scenario, in which the responsible authority performs mystery shopping to test a machine learning service provider. In this scenario, the authority uses the provider's resources and tools to train a model on a given dataset and tests whether the provider included a backdoor. In our proposed approach, the authority creates a reference model pool by training a small number of clean and poisoned models using trusted infrastructure, and calibrates a model distance threshold to identify clean models. We propose and experimentally analyze a number of approaches to compute model distances and we also test a scenario where the provider performs an adaptive attack to avoid detection. The most reliable method is based on requesting adversarial training from the provider. The model distance is best measured using a set of input samples generated by inverting the models in such a way as to maximize the distance from clean samples. With these settings, our method can often completely separate clean and poisoned models, and it proves to be superior to state-of-the-art backdoor detectors as well.

后门检测模型安全语义后门可信验证

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。