用检测引导的遗忘机制,高效清除语言模型中的后门漏洞。
DUP: Detection-guided Unlearning for Backdoor Purification in Language Models
- 通过特征级异常检测识别中毒样本,结合多层距离与转移信息。
- 无需重新训练或外部干净数据,即可实现高精度后门净化。
- 适用于多种攻击和模型架构,特别适合资源受限场景。
随着后门攻击愈发隐蔽和鲁棒,现有防御策略暴露出关键缺陷:检测依赖粗粒度特征统计,净化常需完整重训练或额外干净模型。为此,我们提出DUP(Detection-guided Unlearning for Purification),一个将后门检测与基于遗忘的净化整合的统一框架。检测器通过联合利用类无关距离与层间转移,捕捉特征级异常,以加权方式集成偏差,实现更细粒度分析。基于检测结果,我们采用参数高效的遗忘机制进行模型净化,避免全量重训练且不依赖外部干净模型。具体地,创新性地复用知识蒸馏,使学生模型在检测到的中毒样本上输出与教师模型产生更大分歧,从而有效迫使模型遗忘后门行为。在多种攻击方法与语言模型架构上的大量实验表明,DUP在检测准确率与净化效果上均表现优越。代码已开源:https://github.com/ManHu2025/DUP。
原文摘要 · Abstract (English)
As backdoor attacks become more stealthy and robust, they reveal critical weaknesses in current defense strategies: detection methods often rely on coarse-grained feature statistics, and purification methods typically require full retraining or additional clean models. To address these challenges, we propose DUP (Detection-guided Unlearning for Purification), a unified framework that integrates backdoor detection with unlearning-based purification. The detector captures feature-level anomalies by jointly leveraging class-agnostic distances and inter-layer transitions. These deviations are integrated through a weighted scheme to identify poisoned inputs, enabling more fine-grained analysis. Based on the detection results, we purify the model through a parameter-efficient unlearning mechanism that avoids full retraining and does not require any external clean model. Specifically, we innovatively repurpose knowledge distillation to guide the student model toward increasing its output divergence from the teacher on detected poisoned samples, effectively forcing it to unlearn the backdoor behavior. Extensive experiments across diverse attack methods and language model architectures demonstrate that DUP achieves superior defense performance in detection accuracy and purification efficacy. Our code is available at https://github.com/ManHu2025/DUP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。