六款大模型普遍低估全球新闻自由，尤其高自由国家被严重误判。
The Democratic Paradox in Large Language Models' Underestimation of Press Freedom
- 对比世界新闻自由指数，六模型普遍低评各国新闻自由度。
- 93%国家被判定自由度不足，高自由国遭更严重低估。
- 模型对本国新闻自由有显著偏好，最高偏好评分超预期260%。
随着大型语言模型（LLMs）日益成为全球数百万用户获取信息的主要渠道，其对齐与偏见可能影响公众对新闻自由等基本民主制度的认知与信任。本研究揭示了六款主流大模型在评估180个国家新闻自由状况时存在的三种系统性偏差：模型整体呈现负向误判，将71%至93%的国家评为自由度较低；存在一种悖论式偏差——模型尤其低估新闻自由水平最高的国家；此外，五款模型表现出正向本土偏见，在其母国评分上显著高于基准预期，部分国家评分高出7%至260%。若大模型未来成为主要搜索引擎与文化工具，必须确保对全球人权与公民权利状态的真实呈现。
原文摘要 · Abstract (English)
As Large Language Models (LLMs) increasingly mediate global information access for millions of users worldwide, their alignment and biases have the potential to shape public understanding and trust in fundamental democratic institutions, such as press freedom. In this study, we uncover three systematic distortions in the way six popular LLMs evaluate press freedom in 180 countries compared to expert assessments of the World Press Freedom Index (WPFI). The six LLMs exhibit a negative misalignment, consistently underestimating press freedom, with individual models rating between 71% to 93% of countries as less free. We also identify a paradoxical pattern we term differential misalignment: LLMs disproportionately underestimate press freedom in countries where it is strongest. Additionally, five of the six LLMs exhibit positive home bias, rating their home countries' press freedoms more favorably than would be expected given their negative misalignment with the human benchmark. In some cases, LLMs rate their home countries between 7% to 260% more positively than expected. If LLMs are set to become the next search engines and some of the most important cultural tools of our time, they must ensure accurate representations of the state of our human and civic rights globally.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。