针对乌尔都语缺乏AI文本检测工具的问题,提出首个专用检测框架。
AI-Generated Text Detection in Low-Resource Languages: A Case Study on Urdu
- 构建包含3600条文本的平衡数据集,融合多模型生成与人工撰写内容。
- mDeBERTa-v3-base模型在测试集上达到91.29%的F1分数,性能最优。
- 为低资源语言NLP和防伪信息传播提供实用方案,适合相关研究者参考。
大型语言模型如今能生成极似人类写作的文本,虽有助于内容创作,却使判断文本来源愈发困难,尤其在乌尔都语等低资源语言中更是如此。为此,本文提出一种专为乌尔都语设计的AI生成文本检测框架。构建了一个包含1,800条人工撰写的文本和1,800条由Gemini、GPT-4o-mini和Kimi AI生成的平衡数据集。通过字符与词数、词汇丰富度(类型-标记比)、N-gram模式等特征进行详细语言学与统计分析,并采用t检验和Mann-Whitney U检验评估显著性。在此基础上,对三种主流多语言Transformer模型——mdeberta-v3-base、distilbert-base-multilingualcased、xlm-roberta-base进行了微调。其中,mDeBERTa-v3-base表现最佳,在测试集上获得91.29%的F1分数和91.26%的准确率。该研究推动了乌尔都语社区应对虚假信息与学术不端的努力,并促进了低资源语言NLP工具的发展。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are now capable of generating text that closely resembles human writing, making them powerful tools for content creation, but this growing ability has also made it harder to tell whether a piece of text was written by a human or by a machine. This challenge becomes even more serious for languages like Urdu, where there are very few tools available to detect AI-generated text. To address this gap, we propose a novel AI-generated text detection framework tailored for the Urdu language. A balanced dataset comprising 1,800 humans authored, and 1,800 AI generated texts, sourced from models such as Gemini, GPT-4o-mini, and Kimi AI was developed. Detailed linguistic and statistical analysis was conducted, focusing on features such as character and word counts, vocabulary richness (Type Token Ratio), and N-gram patterns, with significance evaluated through t-tests and MannWhitney U tests. Three state-of-the-art multilingual transformer models such as mdeberta-v3-base, distilbert-base-multilingualcased, and xlm-roberta-base were fine-tuned on this dataset. The mDeBERTa-v3-base achieved the highest performance, with an F1-score 91.29 and accuracy of 91.26% on the test set. This research advances efforts in contesting misinformation and academic misconduct in Urdu-speaking communities and contributes to the broader development of NLP tools for low resource languages.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。