量化会触发语言模型后门,源模型安全不代表部署后安全。
Quantization-Triggered Backdoors in Language Models: Cross-Quantizer Transferability and the Validation--Deployment Gap
- 通过三阶段对抗微调,在通过精度验证的模型中植入隐蔽恶意行为。
- 量化后翻译模型敌友误判率最高达85.02%,立场分类器偏见值上升0.33。
- 攻击在不同量化器间可迁移,比特数不是决定因素,适合安全评估者阅读。
后训练量化常被视为边缘部署大语言模型的语义中立优化。当全精度源检查点经评估后直接进行量化而未同步重新验证时,该流程产生结构化的验证-部署差距:因量化是参数空间上的多对一映射,源精度认证无法保证部署配置下的行为等价性。我们通过量化行为等价类(QBECs)形式化这一差距,并证明QBEC成员不蕴含行为等价性,为量化触发后门攻击提供理论基础。基于三阶段对抗微调框架,我们在满足源精度检查的模型中嵌入潜在恶意载荷,使其在INT8或4比特压缩后激活目标对抗行为。我们在战术机器翻译与政治内容分析两个实际场景中评估该威胁,将先前仅针对解码器的因果语言模型扩展至多语言编码器-解码器序列到序列模型。结果表明,后门翻译模型在修复后的FP16下零误判,量化后敌友误判率最高达85.02%;配对立场分类器测量出意识形态偏移高达ΔBias=0.33。跨量化器迁移分析显示,攻击持续性取决于量化方案与模型架构,而非名义比特位数。这些发现表明,仅靠源精度审计无法排除量化触发行为,最终部署配置必须纳入行为认证以保障边缘AI可信性。
原文摘要 · Abstract (English)
Post-training quantization is often treated as a semantically neutral optimization for edge deployment of Large Language Models. When a full-precision source checkpoint is evaluated and quantization is applied downstream without equivalent re-evaluation, this workflow creates a structural validation--deployment gap: because quantization is a many-to-one mapping over parameter space, source-precision certification does not guarantee behavioral equivalence in the deployed configuration. We formalize this gap through Quantization Behavioral Equivalence Classes (QBECs) and prove that QBEC membership does not imply behavioral equivalence, providing a theoretical basis for quantization-triggered backdoor attacks. Building on a three-stage adversarial fine-tuning framework, we embed latent malicious payloads into models that satisfy the source-precision checks used in our evaluation, yet activate targeted adversarial behavior upon INT8 or 4-bit compression. We evaluate this threat in two operationally motivated scenarios, tactical machine translation and political content analysis, extending prior work from decoder-only causal LMs to multilingual encoder-decoder sequence-to-sequence models. Results show that backdoored translation models move from zero measured friend--foe corruption at repaired FP16 to up to 85.02% inversion after quantization, and that a paired stance classifier measures an ideological shift of up to $Δ\mathrm{Bias}=0.33$ upon compression. A cross-quantizer transferability analysis further shows that attack persistence varies across quantization schemes and model architectures, rather than being determined by nominal bit-width alone. These findings demonstrate that source-precision auditing alone does not rule out quantization-triggered behavior and that the final deployed configuration must be included in behavioral certification for trustworthy edge AI.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。