用新算法逆推语言模型输出，可精准还原输入文本。
GPT, But Backwards: Exactly Inverting Language Model Outputs
- 提出SODA搜索算法，白盒条件下逆推输入文本
- 对10词内输入，自然与随机文本重建率分别达98%和79%
- 输入长度和词表大小影响更大，33M到3B参数模型均适用
重构语言模型未知输入是评估模型安全性的基础审计手段，可用于检测隐藏系统提示盗用、后门探测及私密数据泄露等问题。现有方法依赖不同程度的信息（如输入输出样本、模型参数、中间激活或输出logits），但常无法完全还原目标输入。本文提出稀疏一热离散Adam（SODA）算法，一种基于搜索的逆推方法，在拥有语言模型白盒访问权及其输出时，可准确重建输入文本。实验首次证明在自然语言与随机输入上实现精确逆推可行。SODA在长度不超过10个词的输入上，重建率分别达到98%和79%。此外，我们发现输入长度与词表大小对成功重建概率的影响远大于模型规模本身，因此该方法可扩展至33M至3B参数的语言模型。
原文摘要 · Abstract (English)
The task of reconstructing unknown textual inputs to language models is a fundamental auditing primitive that allows us to assess the model's vulnerability to a range of security issues, including stealing hidden system prompts, detecting backdoors, and leaking private data. Existing inversion works assume access to differing levels of information (e.g. requiring input-output examples, the model parameters, intermediate activations or output logits) but oftentimes fail to fully reconstruct the desired input. In this paper, we present the Sparse One-hot Discrete Adam (SODA) algorithm, a search-based inversion method that can accurately reconstruct the input text, given white-box access to the language model and its output. Our experiments demonstrate for the first time that exact language model inversion is possible on both natural language and random inputs. Indeed, SODA achieves respectively 98% and 79% reconstruction rates on inputs with lengths up to 10 tokens. Furthermore, we show that input length and vocabulary size have a far greater impact on the probability of a successful reconstruction than the size of the language model itself, thus allowing us to scale to models from 33M to 3B parameters.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。