重新检验GPT检测器对非母语者偏见，发现其实际无系统性误判。
Different Time, Different Language: Revisiting the Bias Against Non-Native Speakers in GPT Detectors
- 在捷克语语境下验证非母语者文本的困惑度不更低
- 三种检测器家族均未显示对非母语者的系统性偏差
- 现代检测器已不依赖困惑度，降低误判风险
大语言模型助手自ChatGPT发布后广受欢迎，但其在学术领域的滥用引发关注，因难以区分人工撰写与生成文本。为此开发了自动化检测技术，初步显示有效。然而，先前研究指出这些方法常将非母语者文章误判为生成内容，归因于其较低的困惑度——被视为检测关键特征。本文两年后在捷克语语境下重新审视该结论，发现非母语者捷克语文本的困惑度并不低于母语者。进一步评估三个不同家族的检测器，未发现系统性偏见。最终表明，当前检测器已不再依赖困惑度即可有效运行。
原文摘要 · Abstract (English)
LLM-based assistants have been widely popularised after the release of ChatGPT. Concerns have been raised about their misuse in academia, given the difficulty of distinguishing between human-written and generated text. To combat this, automated techniques have been developed and shown to be effective, to some extent. However, prior work suggests that these methods often falsely flag essays from non-native speakers as generated, due to their low perplexity extracted from an LLM, which is supposedly a key feature of the detectors. We revisit these statements two years later, specifically in the Czech language setting. We show that the perplexity of texts from non-native speakers of Czech is not lower than that of native speakers. We further examine detectors from three separate families and find no systematic bias against non-native speakers. Finally, we demonstrate that contemporary detectors operate effectively without relying on perplexity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。