改进GLTR检测方法，提升对中英文生成文本的识别准确率。
AI-generated Text Detection with a GLTR-based Approach
- 基于GLTR思想构建GPT-2增强模型，优化文本生成概率分析。
- 英文检测达80.19%宏观F1，接近顶尖模型；西班牙语为66.20%。
- 适用于反虚假信息、学术诚信等场景，适合内容安全研究者。
大型语言模型（LLM）推动了自然语言处理应用的发展，但也可能被滥用于制造假新闻、有害内容、身份冒充或学术剽窃等行为，因其生成文本质量高，难以与人类写作区分。GLTR（Giant Language Model Test Room）是由MIT-IBM Watson AI Lab与HarvardNLP联合开发的可视化工具，可基于GPT-2检测机器生成文本，通过高亮词语的概率差异来提示生成来源。但其结果常存在模糊性。本研究针对IberLef-AuTexTification 2023共享任务，在英语和西班牙语上探索改进GLTR的有效方法。实验表明，所提出的基于GLTR的GPT-2模型在英文数据集上取得80.19%的宏观F1分数，仅次于排名第一的模型（80.91%）。在西班牙语数据集上获得66.20%的宏观F1，落后于最优模型4.57个百分点。
原文摘要 · Abstract (English)
The rise of LLMs (Large Language Models) has contributed to the improved performance and development of cutting-edge NLP applications. However, these can also pose risks when used maliciously, such as spreading fake news, harmful content, impersonating individuals, or facilitating school plagiarism, among others. This is because LLMs can generate high-quality texts, which are challenging to differentiate from those written by humans. GLTR, which stands for Giant Language Model Test Room and was developed jointly by the MIT-IBM Watson AI Lab and HarvardNLP, is a visual tool designed to help detect machine-generated texts based on GPT-2, that highlights the words in text depending on the probability that they were machine-generated. One limitation of GLTR is that the results it returns can sometimes be ambiguous and lead to confusion. This study aims to explore various ways to improve GLTR's effectiveness for detecting AI-generated texts within the context of the IberLef-AuTexTification 2023 shared task, in both English and Spanish languages. Experiment results show that our GLTR-based GPT-2 model overcomes the state-of-the-art models on the English dataset with a macro F1-score of 80.19%, except for the first ranking model (80.91%). However, for the Spanish dataset, we obtained a macro F1-score of 66.20%, which differs by 4.57% compared to the top-performing model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。