用机器学习区分侦探小说中的人类与AI写作，准确率超95%。
Using Machine Learning to Distinguish Human-written from Machine-generated Creative Fiction
- 用朴素贝叶斯和多层感知机分类器识别短文本风格差异
- 100词样本分类准确率超95%，远超人类判别能力（<55%）
- 为出版编辑提供轻量级工具，保护原创作者权益
随着ChatGPT等生成式AI的普及，自动检测大语言模型生成的欺骗性文本成为焦点，主要集中于学术抄袭和假新闻。然而，生成式AI对创意写作者的生计乃至文学文化构成威胁，通过训练大模型模仿特定作家风格生成“伪书”可能形成新型剽窃。该问题研究较少。本研究聚焦经典侦探小说，训练机器学习分类器区分人类与AI生成的短篇创意小说。结果表明，朴素贝叶斯与多层感知机分类器在约100词样本上准确率超过95%，显著优于人类判别者（<55%）。我们已部署在线原型工具AI Detective，作为面向编辑与出版商的轻量可靠应用的第一步，旨在保护人类作者的经济与文化贡献。
原文摘要 · Abstract (English)
Following the universal availability of generative AI systems with the release of ChatGPT, automatic detection of deceptive text created by Large Language Models has focused on domains such as academic plagiarism and "fake news". However, generative AI also poses a threat to the livelihood of creative writers, and perhaps to literary culture in general, through reduction in quality of published material. Training a Large Language Model on writers' output to generate "sham books" in a particular style seems to constitute a new form of plagiarism. This problem has been little researched. In this study, we trained Machine Learning classifier models to distinguish short samples of human-written from machine-generated creative fiction, focusing on classic detective novels. Our results show that a Naive Bayes and a Multi-Layer Perceptron classifier achieved a high degree of success (accuracy > 95%), significantly outperforming human judges (accuracy < 55%). This approach worked well with short text samples (around 100 words), which previous research has shown to be difficult to classify. We have deployed an online proof-of-concept classifier tool, AI Detective, as a first step towards developing lightweight and reliable applications for use by editors and publishers, with the aim of protecting the economic and cultural contribution of human authors.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。