通过词元连贯性提升零样本文本生成检测效果
Zero-Shot Detection of LLM-Generated Text using Token Cohesiveness
- 利用词元连贯性作为新特征，无需训练即可检测大模型生成文本
- 实验证明大模型文本的词元连贯性显著高于人类写作
- 方法适用于黑盒场景，可无缝集成到现有检测器中
大型语言模型（LLMs）能力日益增强且应用广泛，自动识别其生成文本的需求愈发迫切。零样本检测器因其无需训练而受到广泛关注并取得显著成果。本文发现一种新特征——词元连贯性，能有效区分生成与人工文本：大模型生成文本的词元连贯性普遍高于人类写作。基于此，我们提出TOCSIN，一种通用双通道检测范式，将词元连贯性作为即插即用模块，可提升现有零样本检测器性能。计算该特征仅需少量随机删除词元并测量语义差异，特别适合源模型不可访问的黑盒场景。在多个数据集、源模型和评估设置下，对四种顶尖基础检测器的实验表明，该方法具有高效性和通用性。代码已开源。
原文摘要 · Abstract (English)
The increasing capability and widespread usage of large language models (LLMs) highlight the desirability of automatic detection of LLM-generated text. Zero-shot detectors, due to their training-free nature, have received considerable attention and notable success. In this paper, we identify a new feature, token cohesiveness, that is useful for zero-shot detection, and we demonstrate that LLM-generated text tends to exhibit higher token cohesiveness than human-written text. Based on this observation, we devise TOCSIN, a generic dual-channel detection paradigm that uses token cohesiveness as a plug-and-play module to improve existing zero-shot detectors. To calculate token cohesiveness, TOCSIN only requires a few rounds of random token deletion and semantic difference measurement, making it particularly suitable for a practical black-box setting where the source model used for generation is not accessible. Extensive experiments with four state-of-the-art base detectors on various datasets, source models, and evaluation settings demonstrate the effectiveness and generality of the proposed approach. Code available at: \url{https://github.com/Shixuan-Ma/TOCSIN}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。