提出首个针对SAM3的通用跨概念对抗攻击,能有效破坏其文本引导分割能力。
Universal Concept Disruption for SAM3 Image Segmentation

- 设计统一扰动策略,同时干扰文本输入、视觉特征和掩码输出
- 在多个数据集上使平均掩码AP从59.43降至18.73,cgF1从50.32降至20.49
- 扰动具有强迁移性,可适用于SAM3.1及视频推理,适合安全评估使用
SAM3将可提示分割从几何驱动扩展到开放词汇概念分割,通过文本条件接地模型判断概念是否存在并分割所有匹配实例。尽管这种存在门控设计提升了概念级预测性能,其对抗鲁棒性仍未知。本文提出首个专为SAM3设计的通用跨概念对抗攻击——通用概念扰乱(UCD)。UCD从(图像,名词短语)对中学习单一有界图像扰动,以集成概念接地系统的方式攻击SAM3。它联合干扰文本条件输入路径,最大化提示共享视觉特征的差异,抑制最终存在门控的概念得分,并通过面积坍缩和干净掩码Dice破坏扰乱空间有效性。在SACo-Gold、LVIS、RefCOCO、PhraseCut和OpenImages数据集上,UCD在相同评估协议下持续优于所有基线,使平均掩码AP从59.43降至18.73,平均cgF1从50.32降至20.49。所学扰动还可无需重新优化地迁移到SAM3.1及SAM3视频推理,而提示集成、轻量头部微调和时间过滤仅提供有限恢复。
原文摘要 · Abstract (English)
SAM3 extends promptable segmentation from geometry-driven mask prediction to open-vocabulary concept segmentation, where a text-conditioned grounding model decides whether a concept is present and segments all matching instances. While this presence-gated design improves concept-level prediction, its adversarial robustness remains unexplored. In this paper, we introduce Universal Concept Disruption (UCD), the first universal cross-concept adversarial attack tailored to SAM3 image segmentation. UCD learns a single bounded image perturbation from (image, noun-phrase) pairs and attacks SAM3 as an integrated concept-grounding system. It jointly disrupts the text-conditioned input path, maximizes divergence in prompt-shared visual features, suppresses the final presence-gated concept scores, and corrupts the spatial validity of retained masks through area collapse and clean-mask Dice disruption. Across SACo-Gold, LVIS, RefCOCO, PhraseCut, and OpenImages datasets, UCD consistently outperforms all baselines under a matched evaluation protocol, reducing average mask AP from 59.43 to 18.73 and average cgF1 from 50.32 to 20.49. The learned perturbation also transfers to SAM3.1 and to SAM3 video inference without re-optimization, while prompt ensembling, lightweight head fine-tuning, and temporal filtering provide limited recovery.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。