测试模型在多领域生成文本上的检测能力，准确率超99%。
GenAI Content Detection Task 3: Cross-Domain Machine-Generated Text Detection Challenge
- 基于新发布的RAID基准，评估跨领域生成文本检测性能。
- 9支队伍23个提交，多数模型在测试中准确率达99%以上。
- 适合关注生成内容安全与检测技术的研究者参考。
近期多项共享任务聚焦于大语言模型（LLMs）生成文本的检测，但大多局限于单一领域或测试时可能遇到未见领域的场景。本共享任务利用新发布的RAID基准，旨在评估模型能否在已知的大量固定领域和模型生成文本上实现有效检测。任务持续三个月，共有9个团队提交23个检测器。结果显示，多个参赛者在保持5%假阳性率的前提下，对RAID数据集中的机器生成文本检测准确率超过99%，表明当前检测器可稳健应对多领域、多模型生成文本的联合检测。本文探讨该结果的潜在解释，并提出未来研究方向。
原文摘要 · Abstract (English)
Recently there have been many shared tasks targeting the detection of generated text from Large Language Models (LLMs). However, these shared tasks tend to focus either on cases where text is limited to one particular domain or cases where text can be from many domains, some of which may not be seen during test time. In this shared task, using the newly released RAID benchmark, we aim to answer whether or not models can detect generated text from a large, yet fixed, number of domains and LLMs, all of which are seen during training. Over the course of three months, our task was attempted by 9 teams with 23 detector submissions. We find that multiple participants were able to obtain accuracies of over 99% on machine-generated text from RAID while maintaining a 5% False Positive Rate -- suggesting that detectors are able to robustly detect text from many domains and models simultaneously. We discuss potential interpretations of this result and provide directions for future research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。