arXiv:2410.14231cs.CL2024-10被引 5

提出多层级细粒度检测框架,精准识别大模型生成文本

Unveiling Large Language Models Generated Texts: A Multi-Level Fine-Grained Detection Framework

  • 融合结构、语义、语言三层次特征,逐句分析词法语法
  • 对比学习增强对改写文本的识别能力,准确率达88.56%
  • 适合学术机构、出版方用于查伪和作者身份验证

大语言模型(LLMs)在语法修正、内容扩展和文风润色方面显著提升人类写作质量,但其广泛应用也引发作者归属、原创性与伦理问题,威胁学术诚信。现有检测方法多依赖单一特征与二分类,难以有效识别学术场景中的大模型生成文本。为此,本文提出多层级细粒度检测(MFD)框架,融合低层结构、高层语义与深层语言特征,并进行句子级词汇、语法、句法评估。为提升对改写文本的敏感度,采用两种主流规避技术生成变体文本,与原始文本共同训练文本编码器,通过对比学习提取高阶语义特征以增强泛化能力。进一步利用先进大模型分析全文,提取深层语言特征,有效捕捉复杂模式与上下文信息。在公开数据集上的实验表明,MFD模型在多项指标上优于现有方法,实现0.1346的平均绝对误差(MAE)与88.56%的准确率。本研究为机构与出版方提供有效检测机制,助力防范作者身份风险,帮助教育者与编辑优化审核与防剽窃流程。

原文摘要 · Abstract (English)

Large language models (LLMs) have transformed human writing by enhancing grammar correction, content expansion, and stylistic refinement. However, their widespread use raises concerns about authorship, originality, and ethics, even potentially threatening scholarly integrity. Existing detection methods, which mainly rely on single-feature analysis and binary classification, often fail to effectively identify LLM-generated text in academic contexts. To address these challenges, we propose a novel Multi-level Fine-grained Detection (MFD) framework that detects LLM-generated text by integrating low-level structural, high-level semantic, and deep-level linguistic features, while conducting sentence-level evaluations of lexicon, grammar, and syntax for comprehensive analysis. To improve detection of subtle differences in LLM-generated text and enhance robustness against paraphrasing, we apply two mainstream evasion techniques to rewrite the text. These variations, along with original texts, are used to train a text encoder via contrastive learning, extracting high-level semantic features of sentence to boost detection generalization. Furthermore, we leverage advanced LLM to analyze the entire text and extract deep-level linguistic features, enhancing the model's ability to capture complex patterns and nuances while effectively incorporating contextual information. Extensive experiments on public datasets show that the MFD model outperforms existing methods, achieving an MAE of 0.1346 and an accuracy of 88.56%. Our research provides institutions and publishers with an effective mechanism to detect LLM-generated text, mitigating risks of compromised authorship. Educators and editors can use the model's predictions to refine verification and plagiarism prevention protocols, ensuring adherence to standards.

文本检测大模型安全学术诚信

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。