无需标签即可检测语言模型微调中的隐蔽污染,提前7.5倍发现潜在危害。
CANARY: Zero-Label Detection of Fine-Tuning Contamination in Language Models
- 通过两次前向传播分析隐藏状态差异,用稀疏自编码器过滤噪声,直接捕捉语义漂移。
- 在1%污染时达到AUROC=1.000,比输出级方法早7.5倍触发警报。
- 适用于模型治理全链路:检测、验证、优先级排序与修复,适合安全评估团队使用。
攻击者仅需污染1%的微调数据,即可在模型中植入潜在有害行为。这种污染在输出层面完全不可见:有害行为潜伏于模型的隐藏状态几何结构中,直到污染比例超过7.5%才显现。我们提出CANARY(Contamination Auditor via Neural Activation Representation Yield),一种无需标签的检查点审计工具,通过两次对无标签提示集的前向传播,直接从隐藏状态中检测这一隐性变化。CANARY利用稀疏自编码器(SAE)投影隐藏状态差异,滤除风格噪声,提取有意义的语义漂移。在四种模型架构和两种训练范式下,1%污染时达到AUROC=1.000(95%置信区间[0.997, 1.000];Cohen's d=3.28),比任何输出级方法早7.5倍触发警报,且在良性微调下零误报,对风格匹配与梯度噪声自适应攻击具有完全鲁棒性。相同的SAE特征基底可构建完整治理流程:经SAE过滤的放大机制使潜在危害暴露速度提升5倍;按得分排序的提示带来4.2倍红队测试提升;推理时抑制少数污染特异性特征,可将危害从70%降至10%,且无困惑度损失。CANARY是首个仅依赖隐藏状态实现检测、验证、优先级排序与修复的零标签框架。
原文摘要 · Abstract (English)
Adversaries can implant latent harmful behavior by poisoning as few as 1% of fine-tuning examples. The contamination is invisible to every output-level defense: harmful behavior lies dormant in the model's hidden-state geometry and does not appear in generated text until contamination exceeds 7.5%. We introduce CANARY (Contamination Auditor via Neural Activation Representation Yield), a zero-label checkpoint auditor that detects this hidden shift directly from two forward passes over an unlabeled prompt set. CANARY projects the hidden-state difference through a Sparse Autoencoder, filtering style noise to isolate meaningful semantic drift. It achieves AUROC = 1.000 at 1% contamination (95% CI = [0.997, 1.000]; Cohen's d = 3.28) across four model architectures and two training paradigms, 7.5x below where any output-level method fires, with zero false positives on benign fine-tuning and full robustness to style-matching and gradient-noise adaptive attacks. The same SAE feature basis drives a complete governance pipeline: SAE-filtered amplification surfaces latent harm at a 5x higher rate than standard generation; score-ranked prompts yield 4.2x red-teaming lift; and suppressing a handful of contamination-specific features at inference time reduces harm from 70% to 10% with no perplexity penalty. CANARY is the first zero-label framework to detect, verify, prioritize, and remediate supply-chain contamination from hidden states alone.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。