arXiv:2605.20761cs.CL2026-05

评测主流AI文本检测方法,发现识别人类与AI写作较易,但识别具体模型难。

Findings of the Counter Turing Test: AI-Generated Text Detection

  • 用微调Transformer和集成学习提升检测效果。
  • 二分类任务最高F1达1.0000,模型溯源任务最高0.9531。
  • 适合关注AI内容安全与检测技术的研究者。

大型语言模型生成流畅、语境连贯的文本能力不断增强,给数字内容真实性验证系统带来巨大压力。GPT-4、Claude 3.5、Llama等先进生成模型可产出高度自然的人类风格文本,使区分人写与AI生成内容愈发困难。尽管这些模型应用前景广阔,其滥用也引发虚假信息、偏见叙事与安全威胁担忧。本文对当前最先进的AI生成文本检测技术进行了全面分析,并通过“反图灵测试”(CT2)共享任务进行评估。任务A(二分类)要求区分人类写作与AI生成文本,任务B(模型归属)则聚焦于识别具体生成模型。结果显示,二分类任务表现优异,最优系统F1得分达1.0000;而模型归属任务表现显著下降,最佳系统得分0.9531,凸显该任务复杂性。顶尖团队采用微调Transformer、集成学习与混合检测方法,其中DeBERTa与BART基线模型表现突出。然而模型归属任务较低分数表明,区分不同大模型输出仍具挑战,亟需在对抗鲁棒性、特征提取与跨领域泛化方面深入研究。

原文摘要 · Abstract (English)

The growing capability of large language models to produce fluent, contextually coherent text has created mounting pressure on the systems and institutions responsible for ensuring the authenticity of digital content. Advanced generative models such as GPT-4, Claude 3.5, and Llama can produce highly coherent and human-like text, making it increasingly difficult to differentiate between human-written and AI-generated content. While these models have transformative applications, their misuse has raised concerns about misinformation, biased narratives, and security threats. This paper provides a comprehensive analysis of state-of-the-art AI-generated text detection techniques and evaluates their effectiveness through the Counter Turing Test (CT2) shared tasks. Task A (Binary Classification) required participants to distinguish between human-written and AI-generated text, while Task B (Model Attribution) focused on identifying the specific language model responsible for generating a given text. The results demonstrated high performance in binary classification, with the top system achieving an F1 score of 1.0000, but significantly lower scores in model attribution, where the best system achieved 0.9531, highlighting the increased complexity of this task. The top-performing teams leveraged fine-tuned transformer models, ensemble learning, and hybrid detection approaches, with DeBERTa-based and BART-based methods demonstrating strong results. However, the lower scores in Task B underscore the challenges of distinguishing outputs from different LLMs, necessitating further research into adversarial robustness, feature extraction, and cross-domain generalization.

文本检测大模型反图灵测试

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。