测试大模型能否准确识别冲突是源于认知差异还是价值观分歧。
LLMs as mediators: Can they diagnose conflicts accurately?
- 用情景实验对比大模型与人类对因果与道德分歧的辨别能力。
- 大模型普遍高估因果分歧、低估道德分歧，尤其GPT-4在具体表述下更明显。
- 为未来用AI辅助调解冲突提供初步实证依据，适合关注AI伦理与应用的研究者。
先前研究指出，要有效调解分歧，观察者必须能可靠区分分歧是源于对事实认知的不同（因果性）还是价值观的差异（道德性）。本文复现了Koçak等（2003）的情景实验设计，使用OpenAI的GPT 3.5和GPT 4进行测试。结果表明，两类大模型在语义理解上与人类相似，能可靠区分因果与道德分歧。但在诊断对话中的分歧根源时，相比人类，两者均倾向于高估因果分歧、低估道德分歧，尤其在使用依赖具体语言的近端尺度时，GPT 4表现更为显著。GPT 3.5在近端与远端尺度上的表现均不如GPT 4或人类。本研究首次检验了大模型通过识别因果与评价性分歧根源来调解冲突的潜力。
原文摘要 · Abstract (English)
Prior research indicates that to be able to mediate conflict, observers of disagreements between parties must be able to reliably distinguish the sources of their disagreement as stemming from differences in beliefs about what is true (causality) vs. differences in what they value (morality). In this paper, we test if OpenAI's Large Language Models GPT 3.5 and GPT 4 can perform this task and whether one or other type of disagreement proves particularly challenging for LLM's to diagnose. We replicate study 1 in Koçak et al. (2003), which employes a vignette design, with OpenAI's GPT 3.5 and GPT 4. We find that both LLMs have similar semantic understanding of the distinction between causal and moral codes as humans and can reliably distinguish between them. When asked to diagnose the source of disagreement in a conversation, both LLMs, compared to humans, exhibit a tendency to overestimate the extent of causal disagreement and underestimate the extent of moral disagreement in the moral misalignment condition. This tendency is especially pronounced for GPT 4 when using a proximate scale that relies on concrete language specific to an issue. GPT 3.5 does not perform as well as GPT4 or humans when using either the proximate or the distal scale. The study provides a first test of the potential for using LLMs to mediate conflict by diagnosing the root of disagreements in causal and evaluative codes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。