让生成的对抗补丁在不同视角下仍有效,提升物理世界攻击稳定性。
AngleRoCL: Angle-Robust Concept Learning for Physically View-Invariant T2I Adversarial Patches
- 通过学习文本嵌入中的通用概念,指导扩散模型生成抗视角变化的补丁。
- 在多视角测试中,攻击成功率平均提升超50%,显著优于基线方法。
- 适用于研究物理攻击与文本概念关系,对检测系统安全评估有参考价值。
前沿工作已证明,文本到图像(T2I)扩散模型可生成物理世界中误导先进目标检测器的对抗补丁,暴露检测器的脆弱性与风险。然而,现有方法忽视了补丁在物理世界中从不同视角观察时的攻击有效性问题(即视角鲁棒性)。本文系统研究了T2I对抗补丁的视角鲁棒性,揭示其存在显著的视角敏感问题,并发现文本内容对补丁视角鲁棒性影响显著,而任务特定语言指令无法有效提升鲁棒性。为此,我们提出角鲁棒概念学习(AngleRoCL),一种简单灵活的方法,通过学习代表生成角鲁棒补丁能力的通用概念(即文本嵌入),将其融入提示词,引导T2I模型生成天然具备视角不变性的补丁。在五个SOTA检测器上进行的大量仿真与物理实验表明,相较于基线方法,AngleRoCL显著提升了补丁的视角鲁棒性,在多角度条件下攻击成功率平均相对提升超过50%。本研究深化了对物理世界角鲁棒补丁的理解,揭示了文本概念与生成内容物理属性之间的关联。代码已开源:https://github.com/tsingqguo/anglerocl。
原文摘要 · Abstract (English)
Cutting-edge works have demonstrated that text-to-image (T2I) diffusion models can generate adversarial patches that mislead state-of-the-art object detectors in the physical world, revealing detectors' vulnerabilities and risks. However, these methods neglect the T2I patches' attack effectiveness when observed from different views in the physical world (i.e., angle robustness of the T2I adversarial patches). In this paper, we study the angle robustness of T2I adversarial patches comprehensively, revealing their angle-robust issues, demonstrating that texts affect the angle robustness of generated patches significantly, and task-specific linguistic instructions fail to enhance the angle robustness. Motivated by the studies, we introduce Angle-Robust Concept Learning (AngleRoCL), a simple and flexible approach that learns a generalizable concept (i.e., text embeddings in implementation) representing the capability of generating angle-robust patches. The learned concept can be incorporated into textual prompts and guides T2I models to generate patches with their attack effectiveness inherently resistant to viewpoint variations. Through extensive simulation and physical-world experiments on five SOTA detectors across multiple views, we demonstrate that AngleRoCL significantly enhances the angle robustness of T2I adversarial patches compared to baseline methods. Our patches maintain high attack success rates even under challenging viewing conditions, with over 50% average relative improvement in attack effectiveness across multiple angles. This research advances the understanding of physically angle-robust patches and provides insights into the relationship between textual concepts and physical properties in T2I-generated contents. We released our code at https://github.com/tsingqguo/anglerocl.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。