用生成对抗补丁攻击红外可见光行人重识别,让模型认错人。
Generative Adversarial Patches for Physical Attacks on Cross-Modal Pedestrian Re-Identification
- 基于边缘特征自监督训练,生成可物理部署的对抗补丁。
- 在两个数据集上使顶尖模型性能大幅下降,黑盒攻击效果显著。
- 首次针对跨模态行人识别设计物理攻击,适合安全评估研究者。
可见光-红外行人重识别(VI-ReID)旨在匹配由红外与可见光相机拍摄的行人图像。然而,此类跨模态图像匹配任务因以人为中心而面临巨大挑战。现有方法难以提取跨模态公共特征,且在隐式特征空间中弥合差异时丢失关键信息,可能危及安全。本文提出首个针对VI-ReID模型的物理对抗攻击——Edge-Attack。该方法聚焦于跨模态最具辨识度的边缘信息,采用两步策略:首先通过自监督训练多层级边缘特征提取器,捕获个体的判别性边缘表示;其次利用基于视觉变压器生成对抗网络(ViTGAN)的生成模型,生成基于提取边缘特征的对抗补丁。将补丁应用于行人衣物后,生成真实可部署的对抗样本。该黑盒、自监督方法确保了攻击对多种VI-ReID模型的泛化能力。在SYSU-MM01和RegDB数据集上的大量实验,包括真实世界部署,证明Edge-Attack能显著降低当前最先进VI-ReID方法的性能。
原文摘要 · Abstract (English)
Visible-infrared pedestrian Re-identification (VI-ReID) aims to match pedestrian images captured by infrared cameras and visible cameras. However, VI-ReID, like other traditional cross-modal image matching tasks, poses significant challenges due to its human-centered nature. This is evidenced by the shortcomings of existing methods, which struggle to extract common features across modalities, while losing valuable information when bridging the gap between them in the implicit feature space, potentially compromising security. To address this vulnerability, this paper introduces the first physical adversarial attack against VI-ReID models. Our method, termed Edge-Attack, specifically tests the models' ability to leverage deep-level implicit features by focusing on edge information, the most salient explicit feature differentiating individuals across modalities. Edge-Attack utilizes a novel two-step approach. First, a multi-level edge feature extractor is trained in a self-supervised manner to capture discriminative edge representations for each individual. Second, a generative model based on Vision Transformer Generative Adversarial Networks (ViTGAN) is employed to generate adversarial patches conditioned on the extracted edge features. By applying these patches to pedestrian clothing, we create realistic, physically-realizable adversarial samples. This black-box, self-supervised approach ensures the generalizability of our attack against various VI-ReID models. Extensive experiments on SYSU-MM01 and RegDB datasets, including real-world deployments, demonstrate the effectiveness of Edge- Attack in significantly degrading the performance of state-of-the-art VI-ReID methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。