LLM推荐系统普遍存在极化偏见，且对提示词敏感度不一。
Polarization by Default: Auditing Recommendation Bias in LLM-Based Content Curation
- 在真实社交数据上模拟6种提示策略，对比三大厂商模型表现
- 所有配置下均放大信息极化，毒性和情感偏见随提示变化明显
- 适合关注平台内容推荐公平性的研究人员与产品设计者
大型语言模型（LLMs）被越来越多用于内容筛选与排序，但其在该任务中的偏见性质与结构仍不清晰：哪些偏见在不同提供商和平台间具有鲁棒性，哪些可通过提示设计缓解。我们通过受控模拟实验，在推特、蓝星和Reddit的真实社交数据集上，评估了OpenAI、Anthropic、Google三家主流LLM提供商的表现，采用六种提示策略（通用、流行、互动、信息、争议、中立）。在54万次从100篇帖子中选出前10条的模拟中，发现偏见在结构性和提示敏感性上差异显著：所有配置均加剧信息极化；毒性的处理在互动与信息导向提示间呈现显著反转；情感偏见总体偏向负面。模型对比显示：GPT-4o Mini行为最一致；Claude与Gemini在毒性处理上适应性强；Gemini表现出最强的负面情感偏好。在推特上，基于作者资料可推断政治倾向，左倾作者被系统性高估，尽管右倾作者在数据集中占多数，这一模式在多数提示下依然持续。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are increasingly deployed to curate and rank human-created content, yet the nature and structure of their biases in these tasks remains poorly understood: which biases are robust across providers and platforms, and which can be mitigated through prompt design. We present a controlled simulation study mapping content selection biases across three major LLM providers (OpenAI, Anthropic, Google) on real social media datasets from Twitter/X, Bluesky, and Reddit, using six prompting strategies (\textit{general}, \textit{popular}, \textit{engaging}, \textit{informative}, \textit{controversial}, \textit{neutral}). Through 540,000 simulated top-10 selections from pools of 100 posts across 54 experimental conditions, we find that biases differ substantially in how structural and how prompt-sensitive they are. Polarization is amplified across all configurations, toxicity handling shows a strong inversion between engagement- and information-focused prompts, and sentiment biases are predominantly negative. Provider comparisons reveal distinct trade-offs: GPT-4o Mini shows the most consistent behavior across prompts; Claude and Gemini exhibit high adaptivity in toxicity handling; Gemini shows the strongest negative sentiment preference. On Twitter/X, where author demographics can be inferred from profile bios, political leaning bias is the clearest demographic signal: left-leaning authors are systematically over-represented despite right-leaning authors forming the pool plurality in the dataset, and this pattern largely persists across prompts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。