代码大模型会因分词方式泄露高熵秘密，尤其容易记住看似乱码的密钥。
Understanding Secret Leakage Risks in Code LLMs: A Tokenization Perspective
- 发现分词机制导致模型对高字符熵、低词元熵的秘密更易记忆。
- 实验证明部分秘密在模型中记忆率超90%，远高于普通代码片段。
- 适合关注代码安全与分词设计的研究者，尤其涉及隐私泄露防护。
代码密钥是开发者的重要资产，其泄露带来严重网络安全风险。尽管基于代码大语言模型（CLLMs）的AI编程助手快速发展，但研究显示，由于显著的记忆化现象，这些模型会意外泄露敏感信息。本研究首次揭示，字节对编码（BPE）分词方式会导致秘密记忆的异常行为，称之为‘乱码偏差’（gibberish bias）。具体而言，某些密钥虽具有高字符级熵，却因低词元级熵而成为模型最容易记忆的内容。论文通过数值数据验证该偏差存在，指出其根源在于训练数据与密钥数据间词元分布的偏移。进一步分析表明，随着词汇表规模增大，该偏差现象更为突出。最后讨论了缓解策略及对当前分词器设计的深远影响。
原文摘要 · Abstract (English)
Code secrets are sensitive assets for software developers, and their leakage poses significant cybersecurity risks. While the rapid development of AI code assistants powered by Code Large Language Models (CLLMs), CLLMs are shown to inadvertently leak such secrets due to a notorious memorization phenomenon. This study first reveals that Byte-Pair Encoding (BPE) tokenization leads to unexpected behavior of secret memorization, which we term as \textit{gibberish bias}. Specifically, we identified that some secrets are among the easiest for CLLMs to memorize. These secrets yield high character-level entropy, but low token-level entropy. Then, this paper supports the biased claim with numerical data. We identified that the roots of the bias are the token distribution shift between the CLLM training data and the secret data. We further discuss how gibberish bias manifests under the ``larger vocabulary'' trend. To conclude the paper, we discuss potential mitigation strategies and the broader implications on current tokenizer design.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。