让白盒检测方法用大厂模型，实现零样本生成文本识别
Glimpse: Enabling White-Box Methods to Use Proprietary Models for Zero-Shot LLM-Generated Text Detection
- 通过部分输出推断完整概率分布，突破API访问限制
- 在5个主流模型上平均AUROC达0.95，提升51%
- 适合研究生成内容检测或安全防护的开发者
先进大语言模型生成的文本几乎无法与人类写作区分，亟需零样本检测技术。当前方法受限：白盒方法只能用弱开源模型，黑盒方法仅能观察部分输出。本文提出Glimpse，从有限输出中推断完整预测分布，使熵、排名等白盒方法可适配最新专有模型。实验表明，结合Fast-DetectGPT与GPT-3.5时，五种主流模型平均AUROC达0.95，相比开源基线提升51%。结果表明，大模型可有效识别自身输出，或成为对抗自身生成内容的最佳屏障。代码与数据已开源。
原文摘要 · Abstract (English)
Advanced large language models (LLMs) can generate text almost indistinguishable from human-written text, highlighting the importance of LLM-generated text detection. However, current zero-shot techniques face challenges as white-box methods are restricted to use weaker open-source LLMs, and black-box methods are limited by partial observation from stronger proprietary LLMs. It seems impossible to enable white-box methods to use proprietary models because API-level access to the models neither provides full predictive distributions nor inner embeddings. To traverse the divide, we propose **Glimpse**, a probability distribution estimation approach, predicting the full distributions from partial observations. Despite the simplicity of Glimpse, we successfully extend white-box methods like Entropy, Rank, Log-Rank, and Fast-DetectGPT to latest proprietary models. Experiments show that Glimpse with Fast-DetectGPT and GPT-3.5 achieves an average AUROC of about 0.95 in five latest source models, improving the score by 51% relative to the remaining space of the open source baseline. It demonstrates that the latest LLMs can effectively detect their own outputs, suggesting that advanced LLMs may be the best shield against themselves. We release our code and data at https://github.com/baoguangsheng/glimpse.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。