大模型能像人一样判断社交吸引力，且一致性高。
How Do Large Language Models Judge Social Attraction? Evidence from Theory-Grounded Persona Ratings Across Multiple LLMs and Humans
- 用心理学构建三类人格画像，测试大模型评分稳定性
- 大模型与人类对吸引力排序一致，但更极端化
- 无性别呈现差异，适合研究社会认知的自动化评估
大型语言模型（LLMs）越来越多地被用于执行传统上由人类完成的主观评价，但其作为社会评判者的有效性仍不明确。本文通过基于十项心理与关系建构的理论基础人格画像，分为社交吸引、混合及非吸引三类，检验了多模型和人类的判断。研究1中，34个大模型对12个画像重复三次评分，虽个别模型整体倾向不同，但跨轮次稳定性高，三类排序一致，相对顺序高度一致。研究2使用六组匹配的姓名-代词画像对及仅代词的中性名测试，未发现显著性别呈现效应。研究3中198名人类参与者评价相同六组画像，复现三类结构，排序与大模型一致。然而，大模型对吸引型画像评分更高，对非吸引型更低，而两组均未显示性别呈现的显著影响。
原文摘要 · Abstract (English)
Large language models (LLMs) are increasingly used to perform subjective evaluations traditionally made by humans, yet their validity as social judges remains unclear. This paper examines whether LLMs can assess social attraction from theory-grounded persona profiles constructed from ten psychological and relational constructs and organized into three tiers: socially attractive, socially mixed, and socially unattractive. We examine LLM ratings in two studies and compare them with human judgments in a third study. In Study 1, 34 LLMs rated 12 profiles across three repeated runs. Although some models tended to give higher or lower ratings overall, they showed strong stability across runs, consistent three-tier ordering, and high agreement in relative profile ordering. Study 2 examined sensitivity to gender presentation using six matched name-and-pronoun profile pairs and a separate pronoun-only test with a gender-neutral name, finding no significant effects in either analysis. In Study 3, 198 human participants evaluated the six matched profiles from Study 2. Their ratings reproduced the three-tier structure and followed a profile ordering consistent with that of the LLMs. However, LLMs rated attractive profiles more positively and unattractive profiles more negatively than humans, while neither group showed a significant overall effect of gender presentation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。