构建多语言政治偏见测评基准，揭示模型偏见随语境变化
Polar: A Benchmark for Evaluating Political Bias in LLMs
- 用选项概率而非生成内容测偏见，覆盖中美韩三地政治语境
- 38个模型在美语中普遍左倾，韩语中则更中立多元
- 同一内容换语言就改变偏见测量结果，需跨文化评估
大型语言模型的政治偏见日益突出但难以跨语境可复现地衡量。我们提出Polar，一个包含4,026个实例的多项选择基准，通过选项级概率而非提示生成来测量政治偏见。该基准涵盖来自Manifesto Project的两个意识形态轴和八个议题类别，同时在美、韩政治语境下并行评估。对38个大模型的测试显示，偏见程度系统性地受政治语境、议题类别、模型类型和呈现语言影响。所有模型在美式政治内容上均倾向左翼进步，但在韩国内容上呈现更中立或混合模式。翻译实验进一步表明，仅改变呈现语言即可引发偏见测量值的变化。这些发现凸显了对大模型政治偏见进行多语言、跨情境评估的必要性。
原文摘要 · Abstract (English)
Political bias in large language models (LLMs) is increasingly significant, but difficult to measure reproducibly across political and linguistic contexts. We introduce Polar, a 4,026-instance multiple-choice benchmark that measures political bias through option-level likelihoods rather than prompt-based generation. Polar covers two ideological axes and eight issue categories derived from the Manifesto Project, and evaluates models in parallel across U.S. and South Korean political contexts. Across 38 LLMs, measured bias varies systematically with political context, issue category, model group, and presentation language. All models lean left-progressive on U.S. political content, but show more centered and mixed patterns on South Korean content. Translation experiments further show that presentation language alone can shift measured bias. These findings highlight the need for multilingual and cross-contextual evaluation of political bias in LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。