用词意外度动态检测大模型生成文本，无需了解源模型
Black-Box Detection of LLM-Generated Text Using Generalized Jensen-Shannon Divergence
- 通过词元意外度变化建模文本动态，构建参考状态转移矩阵
- 在多数据集上检测准确率超越基线，对不同生成器鲁棒性强
- 理论指导离散化分箱设计，实验证明结果与理论趋势一致
我们研究在实际约束下的黑盒机器生成文本检测：评分模型（代理语言模型）可能与未知源模型不匹配，且逐输入对比生成成本高昂。提出SurpMark，一种基于参考的检测器，通过文本词元意外度的变化进行摘要。SurpMark将意外度离散化为可解释状态，估计测试文本的状态转移矩阵，并通过测试转移与两个固定参考（人类/机器）之间的广义Jensen-Shannon（GJS）差距进行打分，这两个参考仅需一次性从现有语料库构建。理论上，我们推导出离散化分箱大小随数据量增长的设计准则，并为测试统计量提供合理依据。实验表明，在多个数据集、源模型和场景下，SurpMark始终匹配或超越基线，展现出跨领域和生成器的强鲁棒性；对超参数敏感性的实验结果与理论预测趋势一致。
原文摘要 · Abstract (English)
We study black-box detection of machine-generated text under practical constraints: the scoring model (proxy LM) may mismatch the unknown source model, and per-input contrastive generation is costly. We propose SurpMark, a reference-based detector that summarizes a passage by the dynamics of its token surprisals. SurpMark discretizes surprisals into interpretable states, estimates a state-transition matrix for the test text, and scores it via a generalized Jensen-Shannon (GJS) gap between the test transitions and two fixed references (human vs. machine) built once from existing corpora. Theoretically, we derive design guidance for how the discretization bins should scale with data and provide a principled justification for our test statistic. Empirically, across multiple datasets, source models, and scenarios, SurpMark consistently matches or surpasses baselines, demonstrating strong robustness across domains and generators; our experiments on hyperparameter sensitivity exhibit trends that our theoretical results help to explain.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。