提出新型隐空间类别攻击，可让模型误将陌生类当目标类的子类。
A Novel Latent-Class Attack and its Detection by Class Subspace Orthogonalization
- 用未知类样本伪造数据，误导模型将其归为已知类的子类。
- 攻击可绕过访问控制，使敌方被误判为友方。
- 通过类子空间正交化检测异常，无需训练数据即可发现攻击。
深度学习依赖大量训练数据，易受数据投毒攻击，包括通用错误攻击和后门（木马）攻击。本文提出一种新型数据投毒攻击，称为隐空间类别攻击：所有污染样本均来自分类领域中未知的新类别，且被错误标记为该领域中的某个已知类别（目标类别），从而使模型学会将新类别识别为该目标类的子类。此类攻击可用于欺骗基于AI的访问控制系统，或导致‘敌方’被误判为‘友方’。同时，我们提出一种无需访问训练集的后训练防御方法，利用‘类子空间正交化’（CSO）机制，寻找内部表征与所有已知类均不匹配，却仍被高置信度分类到某一已知类的输入样本。针对图像分类任务，还提出可视化估计未知类实例的方法，增强检测结果的可解释性。
原文摘要 · Abstract (English)
Deep learning, which in general relies on voluminous amounts of training data, is vulnerable to data poisoning attacks, including error-generic attacks and backdoors (Trojans). In this work, we propose a new data poisoning attack we dub a latent class attack. Here, all poisoned examples are from a class that is novel (unknown) for the given classification domain and are mislabeled to one of the known classes (the target class) of the domain, so that the model learns to recognize the novel class as a sub-class of the target class. Such attacks could be used e.g. to defeat AI-based access control systems, or could cause a "foe" to be classified as a "friend". We also propose a post-training defense to detect this attack, without any access to the training set. This detection approach builds on "class subspace orthogonalization" (CSO), a plug-and-play paradigm demonstrated to improve existing backdoor detectors. Here, CSO is used to seek an input (a putative unknown class instance) whose internal representation is not aligned with any of the known classes, and yet which is classified with confidence to one of these classes. Finally, specific to image classification domains, we propose a method for visualizing the estimated unknown class instance, providing explainability to our latent class detections.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。