GPT-4可模拟2016年美国大选期间Reddit用户发言,生成难辨真假的政治评论。
How Large Language Models play humans in online conversations: a simulated study of the 2016 US politics on Reddit
- 用GPT-4模仿真实或虚构的立场用户生成评论
- 生成内容在政治倾向和语义上与真人高度相似,但更易制造共识
- 虽能被语义空间区分,但人工难以辨别真伪,提示潜在舆论操控风险
大型语言模型(LLMs)在自然语言生成中表现出强大能力,其模仿人类互动的能力引发关注,尤其在政治敏感的在线讨论中。本研究评估了LLMs在真实、争议性场景——2016年美国大选期间的Reddit讨论中的表现。我们设计三组实验,让GPT-4以真实或虚构的党派用户身份生成评论,分析其政治倾向、情感特征与语言模式,并与真实用户内容对比,同时以零模型为基准。结果表明,GPT-4能生成逼真的评论,支持或反对社区立场皆可,但倾向于促成共识而非分歧。尽管真实与人工评论在语义嵌入空间中可分离,却难以通过人工判断区分。研究揭示了LLMs可能潜入在线讨论,影响政治辩论并塑造叙事,对人工智能驱动的舆论操纵具有广泛警示意义。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have recently emerged as powerful tools for natural language generation, with applications spanning from content creation to social simulations. Their ability to mimic human interactions raises both opportunities and concerns, particularly in the context of politically relevant online discussions. In this study, we evaluate the performance of LLMs in replicating user-generated content within a real-world, divisive scenario: Reddit conversations during the 2016 US Presidential election. In particular, we conduct three different experiments, asking GPT-4 to generate comments by impersonating either real or artificial partisan users. We analyze the generated comments in terms of political alignment, sentiment, and linguistic features, comparing them against real user contributions and benchmarking against a null model. We find that GPT-4 is able to produce realistic comments, both in favor of or against the candidate supported by the community, yet tending to create consensus more easily than dissent. In addition we show that real and artificial comments are well separated in a semantically embedded space, although they are indistinguishable by manual inspection. Our findings provide insights on the potential use of LLMs to sneak into online discussions, influence political debate and shape political narratives, bearing broader implications of AI-driven discourse manipulation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。