通过打乱文本检测机器生成内容，无需训练即可高效区分真假文本。
Luminol-AIDetect: Fast Zero-shot Machine-Generated Text Detection based on Perplexity under Text Shuffling
- 用随机打乱文本引发困惑度变化，捕捉机器生成文本的结构脆弱性。
- 在18种语言、11种攻击类型下，误报率比之前方法低最多17倍。
- 零样本、低成本，适合快速部署到多语言场景中使用。
机器生成文本检测需识别跨模型的结构不变信号，而非依赖特定模型指纹。我们假设尽管大语言模型擅长局部语义一致性，但其自回归特性导致一种特有的结构脆弱性，区别于人类写作。为此提出Luminol-AIDetect，一种新颖的零样本统计方法，通过简单的随机文本打乱过程暴露该脆弱性。实验表明，打乱后困惑度的变化可作为原理性强、模型无关的判别依据：机器生成文本在打乱后困惑度分布显著分散，而人类文本则保持较稳定。该方法从原文及其打乱版本中提取少量困惑度特征，通过密度估计与集成预测完成判断。在8个内容领域、11种对抗攻击类型、18种语言上评估，性能达当前最优，误报率最高降低17倍，且计算成本更低。
原文摘要 · Abstract (English)
Machine-generated text (MGT) detection requires identifying structurally invariant signals across generation models, rather than relying on model-specific fingerprints. In this respect, we hypothesize that while large language models excel at local semantic consistency, their autoregressive nature results in a specific kind of structural fragility compared to human writing. We propose Luminol-AIDetect, a novel, zero-shot statistical approach that exposes this fragility through coherence disruption. By applying a simple randomized text-shuffling procedure, we demonstrate that the resulting shift in perplexity serves as a principled, model-agnostic discriminant, as MGT displays a characteristic dispersion in perplexity-under-shuffling that differs markedly from the more stable structural variability of human-written text. Luminol-AIDetect leverages this distinction to inform its decision process, where a handful of perplexity-based scalar features are extracted from an input text and its shuffled version, then detection is performed via density estimation and ensemble-based prediction. Evaluated across 8 content domains, 11 adversarial attack types, and 18 languages, Luminol-AIDetect demonstrates state-of-the-art performance, with gains up to 17x lower FPR while being cheaper than prior methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。