用下注式序贯检验实现文本生成源的实时精准识别
Online Detection of LLM-Generated Texts via Sequential Hypothesis Testing by Betting
- 基于下注的序贯假设检验，动态判断文本是否由大模型生成
- 在保证误报率可控前提下，平均更快识别出大模型来源
- 适合需实时防伪的新闻、社交平台等在线内容场景
近年来，区分机器生成文本与人工撰写文本的算法备受关注。现有方法多为离线设定，即预先提供混合真实与机器生成文本的数据集，判断每条样本来源。然而，在新闻网站、社交媒体和在线论坛等实际场景中，内容以流式方式持续发布。因此，在在线场景下，如何快速且准确地判断来源是否为大语言模型（LLM），并具备强统计保障，对媒体平台有效运作及防范虚假信息传播至关重要。为此，本文提出一种基于下注式序贯假设检验的算法，不仅可补充现有离线检测方法，还具备统计保障：可控的假阳性率以及正确识别模型来源的期望时间。实验验证了该方法的有效性。
原文摘要 · Abstract (English)
Developing algorithms to differentiate between machine-generated texts and human-written texts has garnered substantial attention in recent years. Existing methods in this direction typically concern an offline setting where a dataset containing a mix of real and machine-generated texts is given upfront, and the task is to determine whether each sample in the dataset is from a large language model (LLM) or a human. However, in many practical scenarios, sources such as news websites, social media accounts, and online forums publish content in a streaming fashion. Therefore, in this online scenario, how to quickly and accurately determine whether the source is an LLM with strong statistical guarantees is crucial for these media or platforms to function effectively and prevent the spread of misinformation and other potential misuse of LLMs. To tackle the problem of online detection, we develop an algorithm based on the techniques of sequential hypothesis testing by betting that not only builds upon and complements existing offline detection techniques but also enjoys statistical guarantees, which include a controlled false positive rate and the expected time to correctly identify a source as an LLM. Experiments were conducted to demonstrate the effectiveness of our method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。