提出首个针对行人属性识别的对抗攻击与防御框架,提升模型鲁棒性。
Adversarial Semantic and Label Perturbation Attack for Pedestrian Attribute Recognition
- 基于CLIP的多模态框架,分块处理图像并融合文本提示进行属性识别。
- 设计语义与标签扰动攻击,在多个数据集上使识别准确率下降超15%。
- 适用于提升行人属性识别系统在真实场景下的安全性和抗干扰能力。
行人属性识别(PAR)是人本研究中的关键任务,近年来得益于深度神经网络的发展取得了显著进展。然而,其潜在脆弱性与抗干扰能力仍待深入探索。为此,本文首次提出针对行人属性识别的对抗攻击与防御框架。具体地,基于预训练的基于CLIP的PAR框架,将输入行人图像划分为不重叠块,并通过投影层映射为特征嵌入;同时,利用提示词将属性集合扩展为句子,经预训练的CLIP文本编码器生成属性特征。采用多模态Transformer融合视觉与文本令牌,再通过前馈网络完成属性识别。在此基础上,引入对抗语义与标签扰动生成对抗噪声,称为ASL-PAR。此外,设计语义偏移防御策略以抑制对抗攻击影响。在数字域(PETA、PA100K、MSP60K、RAPv2)和物理域上的大量实验充分验证了所提攻击与防御策略的有效性。论文源码将在https://github.com/Event-AHU/OpenPAR发布。
原文摘要 · Abstract (English)
Pedestrian Attribute Recognition (PAR) is an indispensable task in human-centered research and has made great progress in recent years with the development of deep neural networks. However, the potential vulnerability and anti-interference ability have still not been fully explored. To bridge this gap, this paper proposes the first adversarial attack and defense framework for pedestrian attribute recognition. Specifically, we exploit both global- and patch-level attacks on the pedestrian images, based on the pre-trained CLIP-based PAR framework. It first divides the input pedestrian image into non-overlapping patches and embeds them into feature embeddings using a projection layer. Meanwhile, the attribute set is expanded into sentences using prompts and embedded into attribute features using a pre-trained CLIP text encoder. A multi-modal Transformer is adopted to fuse the obtained vision and text tokens, and a feed-forward network is utilized for attribute recognition. Based on the aforementioned PAR framework, we adopt the adversarial semantic and label-perturbation to generate the adversarial noise, termed ASL-PAR. We also design a semantic offset defense strategy to suppress the influence of adversarial attacks. Extensive experiments conducted on both digital domains (i.e., PETA, PA100K, MSP60K, RAPv2) and physical domains fully validated the effectiveness of our proposed adversarial attack and defense strategies for the pedestrian attribute recognition. The source code of this paper will be released on https://github.com/Event-AHU/OpenPAR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。