通过分析文本结构特征，提升长篇机器生成内容检测准确率
Discourse Features Enhance Detection of Document-Level Machine-Generated Content
- 引入话语分析框架，捕捉长文本的内在结构特征
- 在多个数据集上实现最高15.5%的检测准确率提升
- 适合需要识别深度改写或长文本生成内容的研究者
大型语言模型（LLM）API 的普及使得机器生成内容（MGC）大量涌现，带来学术抄袭和虚假信息传播等挑战。现有检测方法多依赖表面特征，忽视隐含与结构信息，易被改写或长文本伪装欺骗。为此，本文提出新方法与数据集：在公开数据集 Plagbench 基础上，利用 GPT 与 DIPPER 工具扩展生成了 paraphrased Long-Form Question and Answer（paraLFQA）和 paraphrased Writing Prompts（paraWP）数据集。为更好建模长文本结构，提出 DTransformer 模型，结合 PDTB 进行话语分析预处理，编码结构特征。该方法在 paraLFQA 上比当前最优方法提升 15.5% 绝对准确率，在 paraWP 上提升 4%，在 M4 上提升 1.5%。代码与数据已开源。
原文摘要 · Abstract (English)
The availability of high-quality APIs for Large Language Models (LLMs) has facilitated the widespread creation of Machine-Generated Content (MGC), posing challenges such as academic plagiarism and the spread of misinformation. Existing MGC detectors often focus solely on surface-level information, overlooking implicit and structural features. This makes them susceptible to deception by surface-level sentence patterns, particularly for longer texts and in texts that have been subsequently paraphrased. To overcome these challenges, we introduce novel methodologies and datasets. Besides the publicly available dataset Plagbench, we developed the paraphrased Long-Form Question and Answer (paraLFQA) and paraphrased Writing Prompts (paraWP) datasets using GPT and DIPPER, a discourse paraphrasing tool, by extending artifacts from their original versions. To better capture the structure of longer texts at document level, we propose DTransformer, a model that integrates discourse analysis through PDTB preprocessing to encode structural features. It results in substantial performance gains across both datasets - 15.5% absolute improvement on paraLFQA, 4% absolute improvement on paraWP, and 1.5% absolute improvemene on M4 compared to SOTA approaches. The data and code are available at: https://github.com/myxp-lyp/Discourse-Features-Enhance-Detection-of-Document-Level-Machine-Generated-Content.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。