安全模型在正常训练中会崩溃,因潜在安全结构被破坏。
When Safety Geometry Collapses: Fine-Tuning Vulnerabilities in Agentic Guard Models

- 通过分析安全子空间变化,发现良性微调导致安全边界瓦解。
- 某模型拒绝率从85%降至0%,完全丧失安全判断能力。
- 提出新正则化方法,恢复75%拒答率,适合部署安全系统者参考。
在仅使用良性数据的微调下,守护模型会彻底丧失安全对齐——并非受对抗攻击,而是标准领域专业化所致。我们在三种专用于安全分类的模型(LlamaGuard、WildGuard、Granite Guardian)上验证此现象,发现其根源在于潜在安全几何结构的破坏:即指导分类的有害-良性表示边界被摧毁。通过奇异值分解提取各层安全子空间,并追踪其在良性微调下的演变过程。Granite Guardian 出现完全崩溃——拒绝率从85%降至0%,核相关性(CKA)归零,100%输出变为模糊;严重程度超过以往通用大模型的研究结果,这由‘专业化假设’解释:集中式安全表征效率高但极脆弱。为此我们提出鱼贝斯加权安全子空间正则化(FW-SSR),在训练阶段引入两项机制:(i) 基于对角线费舍尔信息的曲率感知方向权重,(ii) 随任务-安全梯度冲突自适应调整的λ_t。FW-SSR使Granite Guardian恢复75%拒答率(CKA=0.983),并将WildGuard攻击成功率降至3.6%——低于未修改基线,通过主动强化安全子空间而非简单锚定实现。三模型结果表明,结构化表征几何(如CKA、费舍尔得分)比绝对位移指标更能可靠预测安全行为,确立以几何为基础的监控是代理型部署中守护模型评估的必要环节。
原文摘要 · Abstract (English)
A guard model fine-tuned on entirely benign data can lose all safety alignment -- not through adversarial manipulation, but through standard domain specialization. We demonstrate this failure across three purpose-built safety classifiers -- LlamaGuard, WildGuard, and Granite Guardian -- deployed as protection layers in agentic AI pipelines, and show that it originates in the destruction of latent safety geometry: the structured harmful -- benign representational boundary that guides classification. We extract per-layer safety subspaces via SVD on class-conditional activation differences and track how this boundary evolves under benign fine-tuning. Granite Guardian undergoes complete collapse -- refusal rate drops from 85\% to 0\%, CKA falls to zero, and 100\% of outputs become ambiguous -- a severity exceeding prior findings on general-purpose LLMs, explained by the specialization hypothesis: concentrated safety representations are efficient but catastrophically brittle. To mitigate this, we propose Fisher-Weighted Safety Subspace Regularization (FW-SSR), a training-time penalty combining (i) curvature-aware direction weights derived from diagonal Fisher information and (ii) an adaptive $λ_t$ that scales with task-safety gradient conflict. FW-SSR recovers 75\% refusal on Granite Guardian (CKA = 0.983) and reduces WildGuard's Attack Success Rate to 3.6\% -- below the unmodified baseline -- by actively sharpening the safety subspace rather than merely anchoring it. Across all three models, structural representational geometry (CKA, Fisher score) predicts safety behavior more reliably than absolute displacement metrics, establishing geometry-based monitoring as a necessary component of guard model evaluation in agentic deployments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。