自适应检测大模型生成文本，准确率提升最高达37%。
AdaDetectGPT: Adaptive Detection of LLM-Generated Text with Statistical Guarantees
- 通过学习训练数据中的判别函数，增强基于概率的检测方法。
- 在多组数据集和模型组合下，性能优于现有最佳方法，最高提升37%。
- 提供统计保证，适合需要可靠检测的场景如内容审核、学术诚信。
我们研究文本是否由人类或大型语言模型（LLM）生成的问题。现有基于对数似然的概率检测方法依赖于给定源LLM分布下的文本对数概率统计，但仅依赖对数概率可能效果不佳。为此，我们提出AdaDetectGPT——一种新型分类器，能从训练数据中自适应学习一个判别函数，以提升基于对数似然检测器的性能。我们为其真阳性率、假阳性率、真阴性率和假阴性率提供了统计保证。大量数值实验表明，AdaDetectGPT在多种数据集与LLM组合下几乎一致优于当前最优方法，性能提升最高可达37%。代码已开源：https://github.com/Mamba413/AdaDetectGPT。
原文摘要 · Abstract (English)
We study the problem of determining whether a piece of text has been authored by a human or by a large language model (LLM). Existing state of the art logits-based detectors make use of statistics derived from the log-probability of the observed text evaluated using the distribution function of a given source LLM. However, relying solely on log probabilities can be sub-optimal. In response, we introduce AdaDetectGPT -- a novel classifier that adaptively learns a witness function from training data to enhance the performance of logits-based detectors. We provide statistical guarantees on its true positive rate, false positive rate, true negative rate and false negative rate. Extensive numerical studies show AdaDetectGPT nearly uniformly improves the state-of-the-art method in various combination of datasets and LLMs, and the improvement can reach up to 37\%. A python implementation of our method is available at https://github.com/Mamba413/AdaDetectGPT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。