通过追踪有害信息传播路径,精准切断恶意逻辑链,提升大模型安全性。
TraceRouter: Robust Safety for Large Foundation Models via Path-Level Intervention
- 基于注意力差异定位敏感层,用稀疏自编码器分离恶意特征
- 通过零置干预计算特征影响度,识别下游因果路径并选择性抑制
- 在不损害正常功能前提下,显著增强对抗鲁棒性,适合安全强化场景
尽管具备强大能力,大基础模型(LFMs)仍易受对抗操纵。现有防御多依赖“局部性假设”,抑制孤立神经元或特征,但有害语义以跨层分布式电路形式存在,导致局部干预脆弱且损害模型效用。为此,我们提出TraceRouter,一种路径级框架,可追踪并切断非法语义的因果传播路径。该框架分三阶段:(1) 通过分析注意力差异定位敏感起始层;(2) 利用稀疏自编码器(SAEs)与差分激活分析解耦并隔离恶意特征;(3) 借助零置干预生成的特征影响分数(FIS),将这些特征映射至下游因果路径。通过选择性抑制这些因果链,TraceRouter物理上切断有害信息流,同时保留正交计算路径。大量实验表明,TraceRouter显著优于现有最先进基线,在对抗鲁棒性与通用效用间实现更优权衡。代码将公开。警告:本文包含不安全模型输出。
原文摘要 · Abstract (English)
Despite their capabilities, large foundation models (LFMs) remain susceptible to adversarial manipulation. Current defenses predominantly rely on the "locality hypothesis", suppressing isolated neurons or features. However, harmful semantics act as distributed, cross-layer circuits, rendering such localized interventions brittle and detrimental to utility. To bridge this gap, we propose \textbf{TraceRouter}, a path-level framework that traces and disconnects the causal propagation circuits of illicit semantics. TraceRouter operates in three stages: (1) it pinpoints a sensitive onset layer by analyzing attention divergence; (2) it leverages sparse autoencoders (SAEs) and differential activation analysis to disentangle and isolate malicious features; and (3) it maps these features to downstream causal pathways via feature influence scores (FIS) derived from zero-out interventions. By selectively suppressing these causal chains, TraceRouter physically severs the flow of harmful information while leaving orthogonal computation routes intact. Extensive experiments demonstrate that TraceRouter significantly outperforms state-of-the-art baselines, achieving a superior trade-off between adversarial robustness and general utility. Our code will be publicly released. WARNING: This paper contains unsafe model responses.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。