构建开放模型文本检测基准，提升机器生成文本识别与作者溯源能力
OpenTuringBench: An Open-Model-based Benchmark and Framework for Machine-Generated Text Detection and Attribution
- 基于开源大模型构建新型评测基准，涵盖多种复杂文本场景
- 提出对比学习检测框架，在多任务中超越多数现有方法
- 适合研究文本生成安全、模型溯源与对抗检测的学者使用
开源大语言模型（OLLMs）在生成式AI应用中日益普及，带来文本检测新挑战。我们提出OpenTuringBench，一个基于OLLMs的新型基准与框架，用于训练和评估机器生成文本检测器在图灵测试与作者溯源问题上的表现。该基准聚焦代表性OLLMs，包含人类/机器操控文本、跨领域文本及未见模型生成文本等高难度任务。我们还推出了OTBDetector，一种对比学习框架，用于检测并归因基于OLLM的机器生成文本。实验结果表明，OpenTuringBench任务具有显著相关性与多样性难度，所提检测器在各项任务中表现优异，显著优于多数现有检测器。相关资源已发布于Hugging Face仓库：https://huggingface.co/datasets/MLNTeam-Unical/OpenTuringBench。
原文摘要 · Abstract (English)
Open Large Language Models (OLLMs) are increasingly leveraged in generative AI applications, posing new challenges for detecting their outputs. We propose OpenTuringBench, a new benchmark based on OLLMs, designed to train and evaluate machine-generated text detectors on the Turing Test and Authorship Attribution problems. OpenTuringBench focuses on a representative set of OLLMs, and features a number of challenging evaluation tasks, including human/machine-manipulated texts, out-of-domain texts, and texts from previously unseen models. We also provide OTBDetector, a contrastive learning framework to detect and attribute OLLM-based machine-generated texts. Results highlight the relevance and varying degrees of difficulty of the OpenTuringBench tasks, with our detector achieving remarkable capabilities across the various tasks and outperforming most existing detectors. Resources are available on the OpenTuringBench Hugging Face repository at https://huggingface.co/datasets/MLNTeam-Unical/OpenTuringBench
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。