arXiv:2605.06294cs.CLcs.AI2026-05

发现现有文本检测方法因错误平均导致误判,提出改进方案显著提升识别准确率。

Log-Likelihood, Simpson's Paradox, and the Detection of Machine-Generated Text

论文配图:Log-Likelihood, Simpson's Paradox, and the Detection of Machine-Generated Text
图 1 · 摘自论文原文
  • 基于贝叶斯理论,在隐藏空间分区域校准似然得分,避免统计结构混淆
  • 在GPT-5.4文本上,检测器AUROC从0.63提升至0.85,性能显著增强
  • 方法可通用适配各类检测模型,为后续研究提供可复用的改进框架

可靠区分大语言模型生成文本与人类撰写文本具有深远社会意义。当前主流方法依赖似然假设:机器生成文本对检测模型而言应更可能。然而我们发现,区分人类与机器文本的词元级信号在检测模型的隐藏空间中分布不均,若对不同统计结构区域简单平均似然分数,会引发类似辛普森悖论的现象——局部强信号被错误聚合所掩盖。为此,我们引入基于贝叶斯决策理论的本地化校准步骤:先学习隐藏空间位置条件下的分数分布轻量预测器,再聚合校准后的对数似然比。此单一干预在所有基线检测器和数据集上均显著且一致地提升性能。例如,我们校准的Fast-DetectGPT在GPT-5.4文本上,AUROC从0.63升至0.85;新提出的局部校准型DMAP检测器实现全面领先。尽管如此,本研究核心贡献并非新检测器,而是精准诊断现有检测器表现不佳的根本原因,并提供一种与任意词元平均流水线兼容的原理性、模块化修复方案,为社区后续工作奠定基础,自然延伸包括更复杂的分布建模、优化校准策略,以及通过完整贝叶斯最优决策规则结合隐藏空间几何信号进行原则性集成。

原文摘要 · Abstract (English)

The ability to reliably distinguish human-written text from that generated by large language models is of profound societal importance. The dominant approach to this problem exploits the likelihood hypothesis: that machine-generated text should appear more probable to a detector language model than human-written text. However, we demonstrate that the token-level signal distinguishing human and machine text is non-uniform across the hidden space of the detector model, and naively averaging likelihood-based token scores across regions with fundamentally different statistical structure, as most detectors do, causes a form of Simpson's paradox: a strong local signal is destroyed by inappropriate aggregation. To correct for this, we introduce a learned local calibration step grounded in Bayesian decision theory. Rather than aggregating raw token scores, we first learn lightweight predictors of the score distributions conditioned on position in hidden space, and aggregate calibrated log-likelihood ratios instead. This single intervention dramatically and consistently improves detection performance across all baseline detectors and all datasets we consider. For example, our calibrated variant of Fast-DetectGPT improves AUROC from $0.63$ to $0.85$ on GPT-5.4 text, and a locally-calibrated DMAP detector we introduce achieves state-of-the-art performance across the board. That said, our central contribution is not a new detector, but a precise diagnosis of a significant cause of under-performance of existing detectors and a principled, modular remedy compatible with any token-averaging pipeline. This will serve as a foundation for the community to build upon, with natural avenues including richer distributional models, improved calibration strategies, and principled ensembling with hidden-space geometry signals via the full Bayes-optimal decision rule.

文本检测贝叶斯校准大模型安全似然分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。