首个针对SAM2的跨提示通用对抗攻击,突破其提示依赖与帧间语义一致性防御。
Vanish into Thin Air: Cross-prompt Universal Adversarial Attacks for SAM2
- 设计目标扫描策略,将每帧分k区域并随机赋提示,降低优化时对特定提示的依赖。
- 提出双语义偏差框架,同时破坏当前帧语义和连续帧间语义一致性,提升攻击效力。
- 在6个数据集上验证,显著超越现有最先进攻击方法,适用于视频分割鲁棒性评估。
近期研究揭示了图像分割基础模型SAM存在对抗样本漏洞。其后续版本SAM2因具备强大的视频分割泛化能力而备受关注,但其鲁棒性尚未被探索,且现有针对SAM的攻击是否可直接迁移至SAM2仍不明确。本文首先分析现有攻击在SAM与SAM2之间的性能差距,指出由提示方向引导及连续帧间语义纠缠带来的两大挑战。为此,我们提出UAP-SAM2,首个基于双语义偏差驱动的跨提示通用对抗攻击方法。为提升跨提示可迁移性,设计目标扫描策略:将每帧划分为k个区域,每个区域随机分配提示,以减少优化过程中的提示依赖。为增强攻击效果,构建双语义偏差框架,通过扭曲当前帧内部语义并破坏连续帧间语义一致性来优化通用对抗扰动。在六个数据集上的两类分割任务中进行充分实验,结果表明该方法对SAM2具有显著有效性。对比结果显示,UAP-SAM2大幅优于现有最先进攻击方法。
原文摘要 · Abstract (English)
Recent studies reveal the vulnerability of the image segmentation foundation model SAM to adversarial examples. Its successor, SAM2, has attracted significant attention due to its strong generalization capability in video segmentation. However, its robustness remains unexplored, and it is unclear whether existing attacks on SAM can be directly transferred to SAM2. In this paper, we first analyze the performance gap of existing attacks between SAM and SAM2 and highlight two key challenges arising from their architectural differences: directional guidance from the prompt and semantic entanglement across consecutive frames. To address these issues, we propose UAP-SAM2, the first cross-prompt universal adversarial attack against SAM2 driven by dual semantic deviation. For cross-prompt transferability, we begin by designing a target-scanning strategy that divides each frame into k regions, each randomly assigned a prompt, to reduce prompt dependency during optimization. For effectiveness, we design a dual semantic deviation framework that optimizes a UAP by distorting the semantics within the current frame and disrupting the semantic consistency across consecutive frames. Extensive experiments on six datasets across two segmentation tasks demonstrate the effectiveness of the proposed method for SAM2. The comparative results show that UAP-SAM2 significantly outperforms state-of-the-art (SOTA) attacks by a large margin.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。