arXiv:2604.24162cs.CRcs.AI2026-04

不修改模型就能在推理时消除后门,靠的是分析注意力分布的异常集中。

Defusing the Trigger: Tail-Risk-Informed Attention Rebalancing for LLM Backdoor Mitigation

  • 通过识别注意力集中在特定词元的异常模式,动态调整可疑注意力分布。
  • 在四种后门攻击下平均误触发率降至11.5%,比最强基线低7.2个百分点。
  • 无需更新参数或额外计算,适合部署在安全敏感的生产环境。

后门大语言模型在推理时表现出攻击者指定的行为,同时对良性输入保持正常性能。现有缓解方法通常需要参数更新或可信干净数据,或依赖辅助生成与多次模型运行,增加了部署复杂性。在多种后门机制中,成功激活的样本在语义内容词元上的注意力呈现更强的尾部集中特征,优于良性输入和未激活触发词的样本。这一特性提供了样本内控制信号,用于选择性调节可疑注意力动态。我们提出TIARA——一种基于尾部风险的注意力重平衡方法,用于推理时后门缓解。TIARA过滤结构化注意力陷阱,聚合跨行与头的稀疏高浓度事件,并将风险信号转化为选择性内容域功率平滑与自适应注意力质量重分配;约束重构将有效注意力分布写回后再进行值聚合。TIARA无需参数更新、辅助生成、额外目标模型遍历或部署时的干净参考集。我们在密集型、推理导向型及稀疏混合专家型LLM上评估了四种后门范式。在三个模型家族中,TIARA将平均宏观攻击成功率(ASR)降至11.5%,优于最强无更新推理基线7.2个百分点,同时将干净任务退化限制在3.8个百分点以内。标准化剖析显示,相比匹配的急切注意力基线,TIARA增加12.9%端到端延迟;当前未融合路径比融合SDPA慢23.3%。总体而言,TIARA确立了样本条件性注意力重平衡作为缓解注意力集中型后门的实用推理时控制层。

原文摘要 · Abstract (English)

Backdoored large language models (LLMs) exhibit attacker-specified behavior at inference time while retaining normal performance on benign inputs. Existing mitigations often require parameter updates and trusted clean data, or rely on auxiliary generation and repeated model execution, complicating deployment. Across diverse backdoor mechanisms, successful activations exhibit stronger tail concentration in attention over semantic-content tokens than benign inputs and unsuccessful trigger activations. This pattern provides a sample-internal control signal to selectively regulate suspicious attention dynamics. We propose TIARA, a tail-risk-informed attention rebalancing approach for inference-time LLM backdoor mitigation. TIARA filters structural attention sinks, aggregates sparse high-concentration events across rows and heads, and converts the risk signal into selective content-domain power smoothing and adaptive attention-mass reallocation. A constrained reconstruction writes valid attention distributions back before value aggregation. TIARA requires no parameter updates, auxiliary generation, additional target-model passes, or deployment-time clean reference sets. We evaluate TIARA across four backdoor paradigms on dense, reasoning-oriented, and sparse mixture-of-experts LLMs. Across three model families, TIARA reduces average macro ASR to 11.5%, outperforming the strongest no-update inference-time baseline by 7.2 percentage points while limiting clean-task degradation to at most 3.8 percentage points. Under standardized profiling, TIARA adds 12.9% end-to-end latency over a matched eager-attention baseline; the current unfused path is 23.3% slower than fused SDPA. Overall, TIARA establishes sample-conditional attention rebalancing as a practical inference-time control layer for mitigating attention-concentrated LLM backdoors.

后门防御注意力分析推理优化LLM安全

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。