重新定义AI生成文本检测，构建真实协作文本基准。
'Your AI Text is not Mine': Redefining and Evaluating AI-generated Text Detection under Realistic Assumptions
- 提出多种AI生成文本的定义并构建真实协作文本数据集
- 现有检测器仅对特定类型有效，泛化能力差
- 适合关注真实场景下文本检测的研究者和从业者
尽管普遍认为AI生成文本带来广泛社会风险，但当前检测研究对何为有害使用缺乏共识。现有数据集与方法常自行设定标准，假设往往隐含且与现实应用关联松散。为此，本文系统定义了多种AI生成文本的概念及其特征。为研究这些概念，我们构建了AITDNA——一个标注了完整编辑与AI交互历史的人机协作文本新基准。在该基准上评估多种检测器，发现它们仅在特定定义下表现良好，难以作为通用检测工具。相关代码与数据已公开。
原文摘要 · Abstract (English)
Although it is generally agreed that AI-generated text poses a broad societal risk, there is no common understanding in the AI-generated text detection literature on what constitutes harmful use. Rather, existing datasets and approaches often define their own criteria and make their own assumptions, sometimes implicitly, and often only loosely related to real-world needs and applications. To address this gap, we here systematically define various notions of AI-generated text and their characteristics. To study these, we collect AITDNA - a new benchmark of human-machine co-constructed texts that is annotated with detailed genesis information, such as the entire edit and AI-interaction history. We benchmark various machine-generated text detectors and find that they often only perform well for specific notions but not as broad detectors. We release code and data publicly.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。