26个大模型政治倾向多维审计，发现多数偏左自由主义。
Political Alignment in Large Language Models: A Multidimensional Audit of Psychometric Identity and Behavioral Bias
- 用三大心理量表+新闻偏见任务多维度评估模型政治立场。
- 96.3%模型集中在自由左翼区域，模型身份解释力超90%方差。
- 单轴评价不足，需多维框架理解模型对齐行为。
随着大语言模型（LLMs）广泛应用，理解其政治定位对评估对齐性及下游影响至关重要。我们使用三种政治心理量表（Political Compass、SapplyValues、8Values）和新闻偏见标注任务，审计了26个主流LLM。为测试鲁棒性，量表在多种语义提示变体下施测，并通过双因素方差分析分离模型与提示效应。结果显示，多数模型聚集于相似意识形态区域，96.3%位于Political Compass的自由左翼象限，且模型身份解释了超过90%的提示变体间方差（η² > 0.90）。跨量表比较表明，Political Compass的社会轴心与文化进步主义相关性更强（r = -0.64），而与权威类测量关联较弱。开源与闭源模型存在差异，且在检测极端政治偏见任务中表现不对称。回归分析显示，心理量表中的意识形态定位无法显著预测分类错误，未发现对话意识形态与任务行为之间存在统计显著关系。研究提示：单一维度评估不足以刻画对齐行为，多维审计框架对部署模型尤为重要。代码与数据已公开于https://github.com/sakhadib/PolAlignLLM。
原文摘要 · Abstract (English)
As large language models (LLMs) are increasingly deployed, understanding how they express political positioning is important for evaluating alignment and downstream effects. We audit 26 contemporary LLMs using three political psychometric inventories (Political Compass, SapplyValues, 8Values) and a news bias labeling task. To test robustness, inventories are administered across multiple semantic prompt variants and analyzed with a two-way ANOVA separating model and prompt effects. Most models cluster in a similar ideological region, with 96.3% located in the Libertarian-Left quadrant of the Political Compass, and model identity explaining most variance across prompt variants ($η^2 > 0.90$). Cross-instrument comparisons suggest that the Political Compass social axis aligns more strongly with cultural progressivism than authority-related measures ($r=-0.64$). We observe differences between open-weight and closed-source models and asymmetric performance in detecting extreme political bias in downstream classification. Regression analysis finds that psychometric ideological positioning does not significantly predict classification errors, providing no evidence of a statistically significant relationship between conversational ideological identity and task-level behavior. These findings suggest that single-axis evaluations are insufficient and that multidimensional auditing frameworks are important to characterize alignment behavior in deployed LLMs. Our code and data are publicly available at https://github.com/sakhadib/PolAlignLLM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。