发现少选词反而更准，关键在模型强弱与熵校准。
When Less is More: Understanding When Token Filtering Helps and Fails in AI-generated Text Detection
- 用熵差分数筛选前k个高概率词，诊断检测效果
- 保留40%词时检测最优，但只对弱模型有效
- 强模型中低熵词非无害，反成干扰项
大语言模型的快速发展使生成文本检测愈发重要。现有零样本检测器认为更多词级证据带来更高可靠性，但我们的实证研究挑战这一共识：有时少选词反而更好，仅保留40%的词可达到最优性能，且该优势并非普遍适用。通过熵差分数（EGS），我们引入基于前k个累积概率的过滤方法作为诊断工具。在三种典型场景下，过滤行为表现迥异。我们基于典型集理论分析EGS，并通过熵校准与分布分析量化其动态特性。结果表明，过滤对弱源模型有效，因其低熵词具有破坏性；但在强源模型中则无效，因这些词并不显著有害。本工作首次系统揭示：部分词不仅无信息量，更因熵校准偏差而系统性有害，揭示了词级检测中的双重权衡机制。
原文摘要 · Abstract (English)
The rapid advancement of large language models (LLMs) has made AI-generated text detection increasingly critical. Existing zero-shot detectors assume that more token-level evidence leads to more reliable detection. However, our empirical study challenges this consensus: fewer tokens sometimes work better, retaining only 40% can yield optimal performance, yet this benefit is not universal. Using the Entropy Gap Score (EGS), we introduce top-$k$ cumulative probability filtering as a diagnostic probe. Across three representative settings, filtering exhibits strikingly different behaviors. We analyze EGS via typical set theory and quantify its dynamics through entropy calibration and distribution analysis. We find that filtering helps for weak source LMs, where low-entropy tokens are harmful, but fails for strong source LMs, where they are not notably harmful. Our work provides the first systematic analysis showing that some tokens are not merely uninformative but systematically harmful due to entropy miscalibration, revealing a two-sided trade-off in token-level detection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。