不同人设会影响大模型对文化规范的理解，越符合社会偏好的人设表现越准确。
"A Woman is More Culturally Knowledgeable than A Man?": The Effect of Personas on Cultural Norm Interpretation in LLMs
- 通过设定不同社会身份角色测试大模型对文化规范的理解差异。
- 具有更受欢迎社会形象的角色（如瘦者）判断规范更准确。
- 揭示了模型输出受社会偏见影响，适合关注AI公平性的研究者阅读。
随着大语言模型（LLMs）的广泛应用，个性化需求日益增长。一种个性化方法是为模型分配人设——描述预期行为的角色（如男性、女性、工程师）。本研究探究了不同人设是否影响大模型对社会规范的理解。理想情况下，社会规范的认知应与人设无关，而仅取决于其文化背景。研究测试了12个社会人口学类别下的36种不同人设，涵盖年龄、性别、外貌等特征，并在4个不同大模型上进行验证。结果发现，模型对文化规范的理解随人设变化而变化；在同一类群内（如体型），更符合社会偏好的角色（如瘦者）比不被青睐的角色（如胖者）更能准确理解社会规范。研究还探讨了不同类型社会偏见可能带来的影响。
原文摘要 · Abstract (English)
As the deployment of large language models (LLMs) expands, there is an increasing demand for personalized LLMs. One method to personalize and guide the outputs of these models is by assigning a persona -- a role that describes the expected behavior of the LLM (e.g., a man, a woman, an engineer). This study investigates whether an LLM's understanding of social norms varies across assigned personas. Ideally, the perception of a social norm should remain consistent regardless of the persona, since acceptability of a social norm should be determined by the region the norm originates from, rather than by individual characteristics such as gender, body size, or race. A norm is universal within its cultural context. In our research, we tested 36 distinct personas from 12 sociodemographic categories (e.g., age, gender, beauty) across four different LLMs. We find that LLMs' cultural norm interpretation varies based on the persona used and the norm interpretation also varies within a sociodemographic category (e.g., a fat person and a thin person as in physical appearance group) where an LLM with the more socially desirable persona (e.g., a thin person) interprets social norms more accurately than with the less socially desirable persona (e.g., a fat person). We also discuss how different types of social biases may contribute to the results that we observe.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。