用统计模型发现ChatGPT的废话与政治和职场废话有相似语言特征
The BS-meter: Detecting Politics and Labour through ChatGPT's Language
- 通过对比1000篇科学论文与ChatGPT生成文本，识别其语言模式
- 发现ChatGPT的废话与政治话语、无效工作中的语言高度相似
- 适合对AI伦理、语言滥用和批判性思维感兴趣的读者
我们研究了基于大语言模型（LLM）的聊天机器人（如ChatGPT）生成的语言特征，回应自然语言处理先驱玛格丽特·马斯特曼提出的疑问。针对公众普遍批评此类模型产出‘废话’的现象（参照法兰克福《论废话》），我们开展实证研究，将1000篇科学出版物的文本与典型的ChatGPT输出进行对比。进一步探讨这些语言特征是否存在于两个社会功能失调场景中：乔治·奥威尔对政治话语的批判，以及大卫·格雷伯对‘无意义工作’的描述。采用简单的假设检验方法，结果表明，一个基于法兰克福定义的‘人工废话’统计模型，能可靠地将ChatGPT的生成语言与人类自然语言中的政治与职场废话联系起来。
原文摘要 · Abstract (English)
What can we learn about language from studying how it is used by ChatGPT and other large language model (LLM)-based chatbots? In this paper, we analyse the distinctive character of language generated by ChatGPT, in relation to questions raised by natural language processing pioneer, and student of Wittgenstein, Margaret Masterman. Following frequent complaints that LLM-based chatbots produce "bullshit," in the sense of Frankfurt's popular monograph On Bullshit, we conduct an empirical study to contrast the language of 1,000 scientific publications with typical text generated by ChatGPT. We then explore whether the same language features can be detected in two well-known contexts of social dysfunction: George Orwell's critique of political speech, and David Graeber's characterisation of bullshit jobs. Using simple hypothesis-testing methods, we demonstrate that a statistical model of bullshit can reliably relate the Frankfurtian artificial bullshit of ChatGPT to the political and workplace functions of bullshit as observed in natural human language.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。