arXiv:2501.12761cs.CVcs.LG2025-01被引 1

提出统一攻击方法,可同时破坏单模态、跨模态和多模态行人重识别模型。

Modality Unified Attack for Omni-Modality Person Re-Identification

  • 设计模态统一攻击框架,用单一生成器适配多种模态模型。
  • 在多模态代理模型中引入特征扰动损失,提升攻击效果。
  • 适用于安全测试与鲁棒性评估,适合关注模型防御的研究者。

基于深度学习的行人重识别(re-id)模型广泛应用于监控系统。近期研究发现黑箱单模态与跨模态re-id模型易受对抗样本攻击,但多模态re-id模型的鲁棒性尚未被探索。由于无法获知目标黑箱系统部署的具体模型类型,本文提出模态统一攻击方法(MUA),训练特定模态的对抗生成器,以有效攻击涵盖单模态、跨模态和多模态的各类re-id模型。采用多模态模型作为代理模型,先通过度量扰动损失对各模态特征进行扰动再融合。为压缩各类模态模型的共性特征,引入跨模态模拟扰动策略,故意将图像输入非对应模态子网络以模拟跨模态嵌入。此外,设计多模态协同扰动策略,利用多模态特征协同度量扰动损失,全面破坏行人图像中的关键信息。大量实验表明,本方法能有效攻击各类omni-modality re-id模型,在四种场景下平均mAP下降率分别达到55.9%、24.4%、49.0%和62.7%。

原文摘要 · Abstract (English)

Deep learning based person re-identification (re-id) models have been widely employed in surveillance systems. Recent studies have demonstrated that black-box single-modality and cross-modality re-id models are vulnerable to adversarial examples (AEs), leaving the robustness of multi-modality re-id models unexplored. Due to the lack of knowledge about the specific type of model deployed in the target black-box surveillance system, we aim to generate modality unified AEs for omni-modality (single-, cross- and multi-modality) re-id models. Specifically, we propose a novel Modality Unified Attack method to train modality-specific adversarial generators to generate AEs that effectively attack different omni-modality models. A multi-modality model is adopted as the surrogate model, wherein the features of each modality are perturbed by metric disruption loss before fusion. To collapse the common features of omni-modality models, Cross Modality Simulated Disruption approach is introduced to mimic the cross-modality feature embeddings by intentionally feeding images to non-corresponding modality-specific subnetworks of the surrogate model. Moreover, Multi Modality Collaborative Disruption strategy is devised to facilitate the attacker to comprehensively corrupt the informative content of person images by leveraging a multi modality feature collaborative metric disruption loss. Extensive experiments show that our MUA method can effectively attack the omni-modality re-id models, achieving 55.9%, 24.4%, 49.0% and 62.7% mean mAP Drop Rate, respectively.

对抗攻击行人重识别多模态黑箱

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。