攻击者通过修改模型架构植入隐形后门,可远程操控视觉语言模型行为。
Architectural Backdoors in Vision-Language Model Supply Chains via Representation Steering

- 在模型中间表示中加入触发式引导逻辑,不改训练数据也不调参。
- 触发后模型输出偏离正常结果,污染问答、生成、检索等任务表现。
- 适合关注模型供应链安全的研究者与系统部署方参考。
视觉-语言模型(VLMs)正通过第三方分发预训练权重、架构定义、文本编码器和计算图的供应链模式广泛部署。这种复用机制带来了关键的安全信任边界:下游应用不仅继承参数,还可能引入共享模型组件中的可执行恶意行为。本文揭示了一种新型架构后门攻击——通过表示引导,在不污染训练数据、不控制微调、不修改部署提示的前提下,将潜伏的引导逻辑嵌入模型架构。当触发条件存在时,中间表示被引导至攻击者指定目标;无触发时则无影响,保持正常功能。我们在多个VLM家族及下游任务(如视觉问答、文生图、检索、语义响应偏移)上验证该攻击,结果显示其严重破坏完整性、安全性与排序公平性,同时在干净输入下表现正常。我们进一步证明共享模型组件可携带潜伏引导逻辑,并提出一种审计防御机制,聚焦于分析随模型分发的可执行逻辑而非仅检查权重。
原文摘要 · Abstract (English)
Vision--Language Models (VLMs) are increasingly deployed through a model supply chain in which pretrained checkpoints, architecture definitions, text encoders, and exported computation graphs are distributed by third parties and reused across downstream services. This reuse model creates a security-critical trust boundary: VLM deployments inherit not only learned parameters but also executable behavior encoded in shared model artifacts. In this paper, we show that a malicious provider can exploit this trust boundary by embedding architectural backdoors into VLM supply chains through representation steering. Our attack introduces dormant steering logic into the model architecture through a trigger-gated additive modification of an intermediate representation, without poisoning training data, controlling downstream fine-tuning, or modifying prompts at deployment time. When the trigger is absent, the modification reduces to zero and the model follows its normal computation, preserving clean utility. When the trigger is present, a steering direction shifts the internal representation toward an attacker-defined objective. We evaluate the attack across multiple VLM families and downstream tasks, including visual question answering, text-to-image generation, retrieval, and semantic response biasing. The results show that the proposed architectural steering backdoor compromises integrity, safety enforcement, and ranking fairness while preserving normal behavior on clean inputs. We further show that shared VLM artifacts can carry dormant steering logic against downstream services, and we propose an auditing defense that inspects the executable logic distributed with model artifacts rather than only their learned weights.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。