用功能归因检测模型异常,不依赖隐藏空间,跨模态有效。
Mechanistic Anomaly Detection via Functional Attribution

- 将异常检测转为功能归因问题,通过影响函数衡量样本间功能关联。
- 在视觉模型后门检测上平均防御有效性达0.93,优于当前最佳0.83。
- 适用于后门、对抗样本、分布外样本,可区分模型内多种异常机制。
我们常能通过真实标签验证神经网络输出的正确性,却难以判断其内部机制是否正常。机制异常检测(MAD)旨在识别此类情况,但现有方法或依赖易被混淆的隐空间分析,或仅适配特定架构与模态。本文将MAD重构为功能归因问题:评估可信数据集样本对模型输出的解释程度,归因失败即提示异常行为。通过影响函数,在参数空间采样测量测试样本与小参考集之间的功能耦合。在多种异常类型与模态下进行评估:针对视觉模型后门,在BackdoorBench上平均防御有效性评级(DER)达0.93(覆盖七种攻击与四个数据集),优于次优方法(0.83);在大语言模型中,对多种后门类型(包括显式混淆模型)也显著优于基线。此外,该方法还可检测对抗样本与分布外样本,并能在单一模型中区分多种异常机制。结果表明,功能归因是检测部署模型异常行为的有效、跨模态通用工具。
原文摘要 · Abstract (English)
We can often verify the correctness of neural network outputs using ground truth labels, but we cannot reliably determine whether the output was produced by normal or anomalous internal mechanisms. Mechanistic anomaly detection (MAD) aims to flag these cases, but existing methods either depend on latent space analysis, which is vulnerable to obfuscation, or are specific to particular architectures and modalities. We reframe MAD as a functional attribution problem: asking to what extent samples from a trusted set can explain the model's output, where attribution failure signals anomalous behavior. We operationalize this using influence functions, measuring functional coupling between test samples and a small reference set via parameter-space sampling. We evaluate across multiple anomaly types and modalities. For backdoors in vision models, our method achieves state-of-the-art detection on BackdoorBench, with an average Defense Effectiveness Rating (DER) of 0.93 across seven attacks and four datasets (next best 0.83). For LLMs, we similarly achieve a significant improvement over baselines for several backdoor types, including on explicitly obfuscated models. Beyond backdoors, our method can detect adversarial and out-of-distribution samples, and distinguishes multiple anomalous mechanisms within a single model. Our results establish functional attribution as an effective, modality-agnostic tool for detecting anomalous behavior in deployed models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。