不完整的字节分词易被诱导生成幻觉内容，需警惕
Improbable Bigrams Expose Vulnerabilities of Incomplete Tokens in Byte-Level Tokenizers
- 用异常组合的不完整分词构造对抗性提示
- 实验显示幻觉率显著上升，最高达90%降幅
- 适合关注模型安全与分词漏洞的研究者
分词是连接人类可读文本与模型可处理离散标记的关键步骤。然而，近期研究发现分词器可能被利用以引发模型的非预期行为。本文研究了不完整分词——即由字节级字节对编码（BPE）分词产生的无法解码的残缺标记。我们假设此类标记严重依赖相邻标记，在遇到陌生标记时极为脆弱。为验证此弱点，我们提出不可能的二元组：设计出分布外的不完整分词组合以利用其依赖关系。实验表明，这些不可能二元组极易引发幻觉行为。令人惊讶的是，使用替代分词方式后，相同短语的幻觉率大幅降低（在Llama3.1中减少90%）。我们警示字节级BPE分词器可能引入语言模型的盲区。
原文摘要 · Abstract (English)
Tokenization is a crucial step that bridges human-readable text with model-readable discrete tokens. However, recent studies have revealed that tokenizers can be exploited to elicit unwanted model behaviors. In this work, we investigate incomplete tokens, i.e., undecodable tokens with stray bytes resulting from byte-level byte-pair encoding (BPE) tokenization. We hypothesize that such tokens are heavily reliant on their adjacent tokens and are fragile when paired with unfamiliar tokens. To demonstrate this vulnerability, we introduce improbable bigrams: out-of-distribution combinations of incomplete tokens designed to exploit their dependency. Our experiments show that improbable bigrams are significantly prone to hallucinatory behaviors. Surprisingly, the same phrases have drastically lower rates of hallucination (90% reduction in Llama3.1) when an alternative tokenization is used. We caution against the potential vulnerabilities introduced by byte-level BPE tokenizers, which may introduce blind spots to language models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。