不同大模型生成文本的可检测性差异显著，部分模型更难识别。
Which LLMs are Difficult to Detect? A Detailed Analysis of Potential Factors Contributing to Difficulties in LLM Text Detection
- 基于不平衡数据集训练分类器，分析多领域文本检测难度。
- 科学写作和OpenAI系模型生成文本最难被识别。
- 揭示模型架构与训练数据对检测难度的影响，适合内容审核研究者参考。
随着大语言模型日益普及，其生成文本已广泛存在于科研、学术和创作等领域。然而，不同大模型在架构和训练数据上存在差异，导致部分模型生成的内容更难被检测。本文利用涵盖四个写作领域的两个数据集，采用LibAUC库（专为处理不平衡数据设计的深度学习框架）训练AI生成文本分类器。在Deepfake Text数据集上，检测难度因领域而异，科学写作尤为困难；在聚焦学生作文的Rewritten Ivy Panda（RIP）数据集中，OpenAI系列模型生成的文本被分类器识别为人类写作的概率显著更高。我们进一步探讨了影响检测难度的潜在因素。
原文摘要 · Abstract (English)
As LLMs increase in accessibility, LLM-generated texts have proliferated across several fields, such as scientific, academic, and creative writing. However, LLMs are not created equally; they may have different architectures and training datasets. Thus, some LLMs may be more challenging to detect than others. Using two datasets spanning four total writing domains, we train AI-generated (AIG) text classifiers using the LibAUC library - a deep learning library for training classifiers with imbalanced datasets. Our results in the Deepfake Text dataset show that AIG-text detection varies across domains, with scientific writing being relatively challenging. In the Rewritten Ivy Panda (RIP) dataset focusing on student essays, we find that the OpenAI family of LLMs was substantially difficult for our classifiers to distinguish from human texts. Additionally, we explore possible factors that could explain the difficulties in detecting OpenAI-generated texts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。