用随机矩阵理论识别视觉语言模型中的对抗噪声,过滤干扰但不改模型。
EigenShield: Causal Subspace Filtering via Random Matrix Theory for Adversarially Robust Vision-Language Models
- 基于随机矩阵理论的谱分析,区分语义与干扰特征。
- 在多个攻击下将成功率降至10%以下,优于现有方法。
- 无需训练或修改模型,适合各类视觉语言模型部署。
视觉语言模型(VLMs)继承了大语言模型(LLMs)的对抗脆弱性,其多模态特性进一步放大了风险。现有防御方法如对抗训练、输入变换和启发式检测存在计算成本高、依赖架构且易被自适应攻击绕过等问题。本文提出EigenShield,一种推理时防御机制,利用随机矩阵理论量化高维表示中的对抗扰动。不同于依赖经验启发的方法,EigenShield采用尖峰协方差模型检测结构化谱偏差,通过鲁棒性非符合度评分(RbNS)与分位数阈值分离因果特征向量(含语义信息)与相关性特征向量(易受对抗扰动影响)。通过将嵌入投影至因果子空间,实现对抗噪声过滤,无需修改模型参数或对抗训练。该方法架构无关、攻击无关,显著降低攻击成功率,建立谱分析作为传统防御的原理性替代方案。实验表明,EigenShield持续优于所有现有防御,包括对抗训练、UNIGUARD和CIDER。
原文摘要 · Abstract (English)
Vision-Language Models (VLMs) inherit adversarial vulnerabilities of Large Language Models (LLMs), which are further exacerbated by their multimodal nature. Existing defenses, including adversarial training, input transformations, and heuristic detection, are computationally expensive, architecture-dependent, and fragile against adaptive attacks. We introduce EigenShield, an inference-time defense leveraging Random Matrix Theory to quantify adversarial disruptions in high-dimensional VLM representations. Unlike prior methods that rely on empirical heuristics, EigenShield employs the spiked covariance model to detect structured spectral deviations. Using a Robustness-based Nonconformity Score (RbNS) and quantile-based thresholding, it separates causal eigenvectors, which encode semantic information, from correlational eigenvectors that are susceptible to adversarial artifacts. By projecting embeddings onto the causal subspace, EigenShield filters adversarial noise without modifying model parameters or requiring adversarial training. This architecture-independent, attack-agnostic approach significantly reduces the attack success rate, establishing spectral analysis as a principled alternative to conventional defenses. Our results demonstrate that EigenShield consistently outperforms all existing defenses, including adversarial training, UNIGUARD, and CIDER.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。