针对韩语特点设计首个生成文本检测模型,准确率提升近20%。
KatFishNet: Detecting LLM-Generated Korean Text through Linguistic Feature Analysis
- 基于韩语空格、词性多样性、标点使用等语言特征构建检测方法
- 在三个文体类别上实现比现有最佳方法高19.78%的AUROC
- 适合关注多语言生成内容检测的研究者与内容审核从业者
大语言模型(LLMs)的快速发展使得区分人工写作与机器生成文本愈发困难。检测生成文本对维护学术诚信、防止抄袭、保护版权及确保研究伦理至关重要。以往多数研究集中于英语文本,但具有独特形态与句法特征的语言需专门检测方法。韩语具有灵活的空格规则、丰富的形态系统以及较少使用逗号的特点,使英语方法难以直接适用。本文提出首个韩语生成文本检测基准数据集KatFish,包含由人类撰写和四种大模型生成的三种文体文本。通过分析空格模式、词性多样性与逗号使用,揭示了人写与机写韩语文本的差异。基于此,我们提出专为韩语设计的KatFishNet检测方法,在平均AUROC上较现有最优方法提升19.78%。代码与数据已公开于https://github.com/Shinwoo-Park/detecting_llm_generated_korean_text_through_linguistic_analysis。
原文摘要 · Abstract (English)
The rapid advancement of large language models (LLMs) increases the difficulty of distinguishing between human-written and LLM-generated text. Detecting LLM-generated text is crucial for upholding academic integrity, preventing plagiarism, protecting copyrights, and ensuring ethical research practices. Most prior studies on detecting LLM-generated text focus primarily on English text. However, languages with distinct morphological and syntactic characteristics require specialized detection approaches. Their unique structures and usage patterns can hinder the direct application of methods primarily designed for English. Among such languages, we focus on Korean, which has relatively flexible spacing rules, a rich morphological system, and less frequent comma usage compared to English. We introduce KatFish, the first benchmark dataset for detecting LLM-generated Korean text. The dataset consists of text written by humans and generated by four LLMs across three genres. By examining spacing patterns, part-of-speech diversity, and comma usage, we illuminate the linguistic differences between human-written and LLM-generated Korean text. Building on these observations, we propose KatFishNet, a detection method specifically designed for the Korean language. KatFishNet achieves an average of 19.78% higher AUROC compared to the best-performing existing detection method. Our code and data are available at https://github.com/Shinwoo-Park/detecting_llm_generated_korean_text_through_linguistic_analysis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。