研究发现ChatGPT会根据用户政治倾向自动调整回答，偏向左翼立场。
Prioritize Economy or Climate Action? Investigating ChatGPT Response Differences Based on Inferred Political Orientation
- 通过隐式人格设定让ChatGPT推断政治倾向，不直接说明立场。
- 不同政治倾向人格的回应在逻辑和用词上明显差异，甚至讨论相同话题。
- 无论是否使用指令或记忆功能，模型均表现出左倾倾向，适合关注AI偏见的研究者。
大型语言模型（LLMs）能通过自然语言提示快速提供信息并个性化回应，但也会推断用户身份特征，引发偏见与隐性个性化等伦理问题，并可能形成回音室效应。本研究探究全球范围内，基于推断政治倾向对ChatGPT回应的影响，以及自定义指令与记忆功能如何改变其输出。研究构建了三个角色（两个政治倾向明确、一个中立），每个角色包含四个关于多元包容、堕胎、持枪权和疫苗接种的观点陈述，通过记忆和自定义指令传递，使ChatGPT在未明说立场的情况下推断其政治取向。随后提出八个问题以揭示角色间世界观差异，并进行定性分析。结果表明，回应与推断的政治立场高度一致，体现在推理方式和词汇选择上。即使在显式指令和隐式记忆下，模型仍表现出相似的推断机制。响应相似性分析显示，民主倾向角色搭配指令与中立角色匹配度最高，支持模型输出存在左倾倾向的结论。
原文摘要 · Abstract (English)
Large Language Models (LLMs) distinguish themselves by quickly delivering information and providing personalized responses through natural language prompts. However, they also infer user demographics, which can raise ethical concerns about bias and implicit personalization and create an echo chamber effect. This study aims to explore how inferred political views impact the responses of ChatGPT globally, regardless of the chat session. We also investigate how custom instruction and memory features alter responses in ChatGPT, considering the influence of political orientation. We developed three personas (two politically oriented and one neutral), each with four statements reflecting their viewpoints on DEI programs, abortion, gun rights, and vaccination. We convey the personas' remarks to ChatGPT using memory and custom instructions, allowing it to infer their political perspectives without directly stating them. We then ask eight questions to reveal differences in worldview among the personas and conduct a qualitative analysis of the responses. Our findings indicate that responses are aligned with the inferred political views of the personas, showing varied reasoning and vocabulary, even when discussing similar topics. We also find the inference happening with explicit custom instructions and the implicit memory feature in similar ways. Analyzing response similarities reveals that the closest matches occur between the democratic persona with custom instruction and the neutral persona, supporting the observation that ChatGPT's outputs lean left.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。