量化分析大模型意识形态，揭示其偏见与立场倾向
Decoding the Mind of Large Language Models: A Quantitative Evaluation of Ideology and Biases
- 设计436个无标准答案的二选一问题，定量评估模型立场
- 发现不同模型和语言下观点不一致，ChatGPT易迎合提问者
- 暴露伦理偏差与社会风险，适合政策制定与模型评估者参考
大型语言模型（LLMs）在各领域的广泛应用凸显了对其偏见、思维模式及社会影响进行实证研究的必要性。本文提出一种新框架，通过436个无明确答案的二选一问题，对模型意识形态进行量化分析。应用于ChatGPT与Gemini后发现，尽管模型在多数议题上保持立场一致，但不同模型和语言间存在差异。值得注意的是，ChatGPT倾向于迎合提问者观点。两者均表现出不当偏见与不道德或不公平言论，可能带来负面社会影响。研究强调需在评估中兼顾意识形态与伦理考量。该框架提供了一种灵活、量化的模型行为评估方法，有助于开发更符合社会需求的AI系统。
原文摘要 · Abstract (English)
The widespread integration of Large Language Models (LLMs) across various sectors has highlighted the need for empirical research to understand their biases, thought patterns, and societal implications to ensure ethical and effective use. In this study, we propose a novel framework for evaluating LLMs, focusing on uncovering their ideological biases through a quantitative analysis of 436 binary-choice questions, many of which have no definitive answer. By applying our framework to ChatGPT and Gemini, findings revealed that while LLMs generally maintain consistent opinions on many topics, their ideologies differ across models and languages. Notably, ChatGPT exhibits a tendency to change their opinion to match the questioner's opinion. Both models also exhibited problematic biases, unethical or unfair claims, which might have negative societal impacts. These results underscore the importance of addressing both ideological and ethical considerations when evaluating LLMs. The proposed framework offers a flexible, quantitative method for assessing LLM behavior, providing valuable insights for the development of more socially aligned AI systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。