提出新方法提升跨模型文本生成检测的泛化能力
DetectAnyLLM: Towards Generalizable and Robust Detection of Machine-Generated Text Across Domains and Models
- 采用任务导向的直接差异学习,优化检测器性能
- 在17种大模型上实现超70%的检测性能提升
- 适合需要跨领域、跨模型检测的应用场景
大语言模型快速发展催生了机器生成文本检测的迫切需求。现有方法在复杂真实场景中表现受限:零样本检测依赖模型输出分布,训练型检测易过拟合训练数据,泛化能力差。我们发现训练型检测器的瓶颈源于训练目标与任务需求不一致。为此提出直接差异学习(DDL)策略,直接以任务为导向优化检测器,使其更准确捕捉检测核心语义,从而提升鲁棒性与泛化性。基于此构建统一检测框架DetectAnyLLM,实现多模型、多领域的先进检测性能。为确保评估可靠性,构建MIRAGE——当前最多样化的多任务机器生成文本检测基准。该数据集从5个文本领域中的10个语料库采集人工文本,再由17种前沿大模型重写或修改,涵盖广泛专有模型与文本风格。大量实验表明,现有方法在复杂环境中存在局限;而DetectAnyLLM在相同训练数据和基础评分模型下持续领先,性能提升超70%,验证了DDL的有效性。
原文摘要 · Abstract (English)
The rapid advancement of large language models (LLMs) has drawn urgent attention to the task of machine-generated text detection (MGTD). However, existing approaches struggle in complex real-world scenarios: zero-shot detectors rely heavily on scoring model's output distribution while training-based detectors are often constrained by overfitting to the training data, limiting generalization. We found that the performance bottleneck of training-based detectors stems from the misalignment between training objective and task needs. To address this, we propose Direct Discrepancy Learning (DDL), a novel optimization strategy that directly optimizes the detector with task-oriented knowledge. DDL enables the detector to better capture the core semantics of the detection task, thereby enhancing both robustness and generalization. Built upon this, we introduce DetectAnyLLM, a unified detection framework that achieves state-of-the-art MGTD performance across diverse LLMs. To ensure a reliable evaluation, we construct MIRAGE, the most diverse multi-task MGTD benchmark. MIRAGE samples human-written texts from 10 corpora across 5 text-domains, which are then re-generated or revised using 17 cutting-edge LLMs, covering a wide spectrum of proprietary models and textual styles. Extensive experiments on MIRAGE reveal the limitations of existing methods in complex environment. In contrast, DetectAnyLLM consistently outperforms them, achieving over a 70% performance improvement under the same training data and base scoring model, underscoring the effectiveness of our DDL. Project page: {https://fjc2005.github.io/detectanyllm}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。