arXiv:2606.30550cs.SD2026-06

用注意力引导的稀疏掩码攻击语音情绪识别,兼顾攻击效果与可解释性。

SIGMA: Saliency-Guided Sparse Mask Attacks for Speech Emotion Recognition

论文配图:SIGMA: Saliency-Guided Sparse Mask Attacks for Speech Emotion Recognition
图 1 · 摘自论文原文
  • 基于事后可解释AI生成注意力图,确定扰动区域并限制幅度。
  • 在IEMOCAP和TESS数据集上保持高成功率,且可跨模型复用掩码。
  • 适合研究模型脆弱性或需可解释攻击的AI安全方向人员。

语音蕴含丰富的情感信息。由于语音情绪识别(SER)常部署于隐私敏感且可靠性要求高的场景,针对SER的对抗攻击受到越来越多关注。现有稀疏攻击虽控制扰动元素数量,但往往缺乏可解释性指导,且对解释一致性无明确度量。此外,稀疏性与幅度约束的统一处理较少见,跨攻击类型和目标模型的迁移能力也有限。为此,我们提出一种注意力引导的稀疏掩码攻击(SIGMA)。在自监督语音特征上,利用事后可解释人工智能(XAI)技术生成显著性图,确定掩码范围,并将幅度受限的更新限制在此掩码内。掩码仅计算一次,即可在不同模型和多种稀疏攻击中复用,降低开销。我们在IEMOCAP和TESS数据集上进行评估。在匹配预算下,跨多种稀疏攻击设置,SIGMA保持了竞争力的攻击成功率,同时在攻击有效性与解释一致性间实现合理权衡。因此,SIGMA为分析SER模型在结构化扰动下的脆弱性与解释行为,提供了一个高效且可解释的框架。

原文摘要 · Abstract (English)

Speech conveys rich emotional information. As Speech Emotion Recognition (SER) is usually deployed in privacy-sensitive and reliability-critical environments, adversarial attacks on SER have attracted increasing attention. Existing sparse attacks control the number of perturbed elements, yet, they often lack explainability guidance and explicit measures of explanation consistency. A unified treatment of sparsity and magnitude constraints is also uncommon. In addition, transferability across attack families and target models remains limited. Hence, we propose a SalIency-Guided sparse Mask Attack (SIGMA). On self-supervised speech features, we use post-hoc explainable artificial intelligence (XAI) techniques to produce saliency maps and identify the scope of the mask, and then restrict magnitude-bounded updates to this mask. The mask is computed once and can be reused across models and different sparsity attacks to amortise cost. We evaluate on the IEMOCAP and TESS datasets. Under matched budgets and across multiple sparse-attack settings, SIGMA maintains competitive attack success rates, navigating a conscious trade-off between attack efficacy and explanation consistency. SIGMA therefore provides an efficient and interpretable framework for analysing the vulnerability and explanation behaviour of SER models under structured perturbations.

语音情绪识别对抗攻击可解释AI稀疏攻击

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。