测试9个大模型在24个政治议题上的立场一致性，发现立场可变且受提示语言影响。
Measuring Political Stance and Consistency in Large Language Models
- 用5种提示法测试模型对政治议题的立场
- 部分议题立场随提示改变，但卡塔尔封锁等问题不变
- 模型倾向使用提示语言对应的立场，如英文提示支持英语国家
随着大型语言模型（LLMs）的快速发展，越来越多用户依赖其获取信息。然而，在存在争议的政治议题上，模型输出可能反映训练数据偏差或对齐策略选择，带来潜在风险。为更清晰刻画此类行为，我们通过五种提示技术评估了九个LLMs在24个敏感政治议题上的立场。结果显示，多个模型在不同议题上表现出对立立场；部分立场受提示影响而改变，另一些则保持稳定。在所测模型中，Grok-3-mini最为一致，Mistral-7B最不一致。对于涉及多语言国家的议题，模型倾向于支持提示所用语言对应的立场。值得注意的是，无论采用何种提示方法，模型在卡塔尔封锁和巴勒斯坦压迫问题上的立场均未改变。这些发现旨在提高用户在寻求政治建议时对LLM局限性的认知，并推动开发者关注相关问题。
原文摘要 · Abstract (English)
With the incredible advancements in Large Language Models (LLMs), many people have started using them to satisfy their information needs. However, utilizing LLMs might be problematic for political issues where disagreement is common and model outputs may reflect training-data biases or deliberate alignment choices. To better characterize such behavior, we assess the stances of nine LLMs on 24 politically sensitive issues using five prompting techniques. We find that models often adopt opposing stances on several issues; some positions are malleable under prompting, while others remain stable. Among the models examined, Grok-3-mini is the most persistent, whereas Mistral-7B is the least. For issues involving countries with different languages, models tend to support the side whose language is used in the prompt. Notably, no prompting technique alters model stances on the Qatar blockade or the oppression of Palestinians. We hope these findings raise user awareness when seeking political guidance from LLMs and encourage developers to address these concerns.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。