arXiv:2510.22981cs.AIcs.CV2025-10NeurIPS被引 1

通过降低指令不确定性,生成更高效、可迁移的语义约束对抗样本。

Exploring Semantic-constrained Adversarial Example with Instruction Uncertainty Reduction

  • 设计多维指令不确定性消减框架,稳定语言引导的攻击方向。
  • 首次实现无参考的3D语义约束对抗样本生成,攻击效果显著提升。
  • 适合关注生成式对抗攻击与安全评估的研究者使用。

近期,直接由自然语言指令生成的语义约束对抗样本(SemanticAE)因其灵活的攻击形式成为研究热点。然而,现有方法攻击能力不足,主要因人类指令中的语义不确定性因素(如指代多样性、描述不完整、边界模糊)未被充分探索。为此,本文提出多维度指令不确定性消减(InSUR)框架,生成更具迁移性、适应性和有效性的SemanticAE。在采样维度,提出残差驱动的攻击方向稳定机制,通过粗略预测语言引导采样过程,利用设计的ResAdv-DDIM采样器稳定优化过程,释放多步扩散模型的可迁移与鲁棒攻击能力。在任务建模方面,提出上下文编码的攻击场景约束,补充指令缺失信息;通过引导掩码与渲染器集成,调控2D/3D SemanticAE的约束,增强场景自适应攻击。在生成器评估方面,提出语义抽象化评估增强,明确评估边界,促进更有效的SemanticAE生成器发展。大量实验表明InSUR在迁移攻击性能上表现优越,且首次实现无需参考的3D语义约束对抗样本生成。

原文摘要 · Abstract (English)

Recently, semantically constrained adversarial examples (SemanticAE), which are directly generated from natural language instructions, have become a promising avenue for future research due to their flexible attacking forms. To generate SemanticAEs, current methods fall short of satisfactory attacking ability as the key underlying factors of semantic uncertainty in human instructions, such as referring diversity, descriptive incompleteness, and boundary ambiguity, have not been fully investigated. To tackle the issues, this paper develops a multi-dimensional instruction uncertainty reduction (InSUR) framework to generate more satisfactory SemanticAE, i.e., transferable, adaptive, and effective. Specifically, in the dimension of the sampling method, we propose the residual-driven attacking direction stabilization to alleviate the unstable adversarial optimization caused by the diversity of language references. By coarsely predicting the language-guided sampling process, the optimization process will be stabilized by the designed ResAdv-DDIM sampler, therefore releasing the transferable and robust adversarial capability of multi-step diffusion models. In task modeling, we propose the context-encoded attacking scenario constraint to supplement the missing knowledge from incomplete human instructions. Guidance masking and renderer integration are proposed to regulate the constraints of 2D/3D SemanticAE, activating stronger scenario-adapted attacks. Moreover, in the dimension of generator evaluation, we propose the semantic-abstracted attacking evaluation enhancement by clarifying the evaluation boundary, facilitating the development of more effective SemanticAE generators. Extensive experiments demonstrate the superiority of the transfer attack performance of InSUR. Moreover, we realize the reference-free generation of semantically constrained 3D adversarial examples for the first time.

对抗样本语义约束扩散模型3D生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。