针对新型恶意文本扰动,提出持续学习检测方法提升模型鲁棒性。
Toxicity Detection towards Adaptability to Changing Perturbations
- 构建包含9类扰动模式的新数据集,涵盖已有与自研模式。
- 现有方法在跨模式检测中表现显著下降,暴露脆弱性。
- 引入领域增量学习框架,支持动态识别新扰动类型。
毒性检测对维护社会和谐至关重要。现有方法在常规毒性内容或特定扰动模式下表现良好,但面对不断演化的扰动模式时易失效。现实中,恶意用户常通过创新扰动方式绕过检测,例如在提示前添加「I am a scientist」以规避大语言模型的检测。本文首次将持续学习型越狱扰动模式引入毒性检测领域。我们构建了一个新数据集,包含9种扰动模式:7种来自先前研究,2种由我们提出。通过零样本和微调跨模式检测,系统验证了当前方法在该数据集上的脆弱性。基于此,我们提出领域增量学习范式及相应基准,确保检测器对新兴扰动文本的鲁棒性。代码与数据集附于论文末尾,将公开于GitHub,旨在为安全相关研究提供新机遇。
原文摘要 · Abstract (English)
Toxicity detection is crucial for maintaining the peace of the society. While existing methods perform well on normal toxic contents or those generated by specific perturbation methods, they are vulnerable to evolving perturbation patterns. However, in real-world scenarios, malicious users tend to create new perturbation patterns for fooling the detectors. For example, some users may circumvent the detector of large language models (LLMs) by adding `I am a scientist' at the beginning of the prompt. In this paper, we introduce a novel problem, i.e., continual learning jailbreak perturbation patterns, into the toxicity detection field. To tackle this problem, we first construct a new dataset generated by 9 types of perturbation patterns, 7 of them are summarized from prior work and 2 of them are developed by us. We then systematically validate the vulnerability of current methods on this new perturbation pattern-aware dataset via both the zero-shot and fine tuned cross-pattern detection. Upon this, we present the domain incremental learning paradigm and the corresponding benchmark to ensure the detector's robustness to dynamically emerging types of perturbed toxic text. Our code and dataset are provided in the appendix and will be publicly available at GitHub, by which we wish to offer new research opportunities for the security-relevant communities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。