通过操控分词方式绕过文本分类模型的防护机制
TokenBreak: Bypassing Text Classification Models Through Token Manipulation
- 利用分词策略漏洞，修改输入文本以误导分类模型
- 攻击后目标语言模型仍能理解内容，防御失效
- 可基于模型家族预测脆弱性，适合安全研究人员参考
自然语言处理模型广泛应用于文本分类与生成任务。为完成这些任务，输入文本需经分词转换为模型可理解的形式，从而进行推理与上下文理解。文本分类模型可用于防范大语言模型的提示注入攻击、有害输入及垃圾邮件等网络安全风险。本文提出TokenBreak：一种新型攻击方法，利用分类模型的分词策略缺陷，通过操纵输入文本使模型产生错误分类。重要的是，攻击后的文本仍可被目标语言模型或邮件接收者正常理解并响应，导致本应防范的攻击依然有效。由于分词器与模型架构紧密关联，可根据模型家族预判其是否易受攻击。此外，本文还提出一种无需重新训练即可部署的防御策略，作为额外保护层。
原文摘要 · Abstract (English)
Natural Language Processing (NLP) models are used for text-related tasks such as classification and generation. To complete these tasks, input data is first tokenized from human-readable text into a format the model can understand, enabling it to make inferences and understand context. Text classification models can be implemented to guard against threats such as prompt injection attacks against Large Language Models (LLMs), toxic input and cybersecurity risks such as spam emails. In this paper, we introduce TokenBreak: a novel attack that can bypass these protection models by taking advantage of the tokenization strategy they use. This attack technique manipulates input text in such a way that certain models give an incorrect classification. Importantly, the end target (LLM or email recipient) can still understand and respond to the manipulated text and therefore be vulnerable to the very attack the protection model was put in place to prevent. The tokenizer is tied to model architecture, meaning it is possible to predict whether or not a model is vulnerable to attack based on family. We also present a defensive strategy as an added layer of protection that can be implemented without having to retrain the defensive model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。