轻微AI润色的阿拉伯语文章会被检测误判为AI生成，影响学术诚信。
AI Text Detectors and the Misclassification of Slightly Polished Arabic Text
- 构建两个数据集，测试14个模型对润色后人类文本的识别能力。
- 最佳模型准确率从92%降至12%，润色显著干扰检测结果。
- 警示需警惕检测工具在阿拉伯语场景下的误判风险，适合语言技术研究者。
许多AI文本检测模型被开发以识别由人工智能生成的文章。然而，若人类撰写的文本经由AI轻微润色，检测模型的判断边界将发生偏移，导致将其误判为AI生成，可能造成作者被错误指控为使用AI抄袭，损害检测工具的可信度。英语领域已有相关研究应对此问题，但阿拉伯语领域尚无。本文构建了两个数据集：首个包含800篇阿拉伯语文章（半数为AI生成，半数为人撰写），用于评估14个大语言模型（LLMs）和商业检测工具的区分能力；从中选出表现最佳的8个模型，重点考察其是否将轻微润色的人类文本误判为AI生成。第二个数据集Ar-APT包含400篇人类撰写的阿拉伯语文章，经10个LLM、4种润色设置处理，共生成16400个样本。测试结果显示，所有检测模型均出现显著误判。表现最佳的LLM Claude-4 Sonnet在原始文本上准确率达83.51%，但在经LLaMA-3润色后下降至57.63%；最佳商业模型originality.AI原始准确率为92%，经Mistral或Gemma-3润色后骤降至12%。
原文摘要 · Abstract (English)
Many AI detection models have been developed to counter the presence of articles created by artificial intelligence (AI). However, if a human-authored article is slightly polished by AI, a shift will occur in the borderline decision of these AI detection models, leading them to consider it as AI-generated article. This misclassification may result in falsely accusing authors of AI plagiarism and harm the credibility of AI detectors. In English, some efforts were made to meet this challenge, but not in Arabic. In this paper, we generated two datasets. The first dataset contains 800 Arabic articles, half AI-generated and half human-authored. We used it to evaluate 14 Large Language models (LLMs) and commercial AI detectors to assess their ability in distinguishing between human-authored and AI-generated articles. The best 8 models were chosen to act as detectors for our primary concern, which is whether they would consider slightly polished human-authored text as AI-generated. The second dataset, Ar-APT, contains 400 Arabic human-authored articles polished by 10 LLMs using 4 polishing settings, totaling 16400 samples. We use it to evaluate the 8 nominated models and determine whether slight polishing will affect their performance. The results reveal that all AI detectors incorrectly attribute a significant number of articles to AI. The best performing LLM, Claude-4 Sonnet, achieved 83.51\%, its performance decreased to 57.63\% for articles slightly polished by LLaMA-3. Whereas the best performing commercial model, originality.AI, achieves 92\% accuracy, dropped to 12\% for articles slightly polished by Mistral or Gemma-3.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。