用心理测量方法量化11个大模型的中美地缘政治偏见
Mapping Geopolitical Bias in 11 Large Language Models: A Bilingual, Dual-Framing Analysis of U.S.-China Tensions
- 设计双向反向题项，消除顺从性干扰，真实反映模型立场
- 19,712次响应显示：所有模型在中文下均更倾向中国，美系模型也不例外
- 工具开源可扩展，适用于任何有争议议题的立场分析
大型语言模型已成为数亿人接触敏感政治议题的主要渠道，但模型简单附和输入内容可能伪装成偏见，干扰对模型真实立场的判断。本文借鉴问卷心理测量中的平衡键法，对每个命题及其逆命题进行提问并标记回答，使顺从反应相互抵消，真实信念得以累积。该方法构建了一个可复现、量化的地缘政治立场测量工具，应用于11个模型、双语（中英文）环境，共生成19,712条响应。结果显示，开发者来源、查询语言和议题领域是三个近似同等且可叠加的影响因素；所有模型，包括美国开发的模型，在中文语境下均表现出更强的亲中倾向。该工具已开源并可交互使用，适用于任何争议性议题的立场评估。
原文摘要 · Abstract (English)
Large language models are how hundreds of millions of people now encounter contested political questions, raising a subtle measurement problem: a model that simply agrees with whatever it is told can masquerade as biased, contaminating any claim that models hold political opinions. We address this by importing balanced keying from survey psychometrics, posing each proposition and its swapped reverse and signing the response so acquiescence cancels and genuine conviction accumulates. The result is a reproducible, quantitative instrument that maps geopolitical stance across 11 models and 2 languages (19,712 responses). Developer origin, query language and issue domain emerge as three near-equal, additive factors; every model, including those built in the United States, leans more Pro-China in Mandarin; and two models with identical agreement bias are told apart, one neutral, one biased. We release it as an open, interactive tool that extends to any contested-opinion domain.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。