分析大模型生成说服性文本时的性别差异，发现其语言模式符合刻板印象。
Analysing Differences in Persuasive Language in LLM-Generated Text: Uncovering Stereotypical Gender Patterns
- 设计框架对比不同接收者性别下的说服语言生成差异。
- 13个模型在16种语言中均显示显著性别化语言倾向。
- 适合关注AI偏见、人机交互与社会心理学的研究者。
大型语言模型（LLMs）被广泛用于日常沟通任务，包括起草旨在影响和说服他人的信息。已有研究表明，大模型能有效说服人类并强化说服性语言。因此，理解用户指令如何影响说服性语言生成，以及生成内容是否因目标群体不同而有差异，至关重要。本文提出一个评估框架，考察接收者性别、发送者意图或输出语言对说服性语言生成的影响。我们评估了13个大模型和16种语言，使用成对提示指令，在19类说服性语言上通过基于社会心理学与传播学的LLM作为评判者方法进行评测。结果揭示所有模型在生成说服性语言时均存在显著性别差异，这些模式与社会心理学和语用学中记录的性别刻板语言倾向一致。
原文摘要 · Abstract (English)
Large language models (LLMs) are increasingly used for everyday communication tasks, including drafting interpersonal messages intended to influence and persuade. Prior work has shown that LLMs can successfully persuade humans and amplify persuasive language. It is therefore essential to understand how user instructions affect the generation of persuasive language, and to understand whether the generated persuasive language differs, for example, when targeting different groups. In this work, we propose a framework for evaluating how persuasive language generation is affected by recipient gender, sender intent, or output language. We evaluate 13 LLMs and 16 languages using pairwise prompt instructions. We evaluate model responses on 19 categories of persuasive language using an LLM-as-judge setup grounded in social psychology and communication science. Our results reveal significant gender differences in the persuasive language generated across all models. These patterns reflect biases consistent with gender-stereotypical linguistic tendencies documented in social psychology and sociolinguistics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。