研究人格特质如何影响大模型谈判表现,提升AI在复杂协作中的可靠性。
Exploring Big Five Personality and AI Capability Effects in LLM-Simulated Negotiation Dialogues
- 用因果发现法分析人格对谈判结果的影响
- 宜人性与外向性显著影响可信度和目标达成率
- 适合军事、跨团队协作等高风险场景的AI设计
本文提出一个评估代理型AI系统在关键任务谈判场景中的框架,解决其需适应多样人类操作者与利益相关方的需求。基于Sotopia模拟测试平台,开展两项实验:实验一通过因果发现方法,量化人格特质对价格谈判的影响,发现宜人性与外向性显著影响可信度、目标达成与知识获取;从团队沟通中提取的社会认知词汇指标,可检测出代理在共情表达、道德基础与观点模式上的细微差异,为高风险操作场景下的可靠AI系统提供可操作洞察。实验二通过操控模拟人类人格与AI特性(透明度、能力、适应性),评估人机工作谈判,揭示AI可信度对任务效能的影响。研究成果建立了一种可重复的评估方法,支持在不同操作者人格与人机协作动态下测试AI可靠性,直接满足复杂任务中可靠AI系统的实际需求。本工作推动了代理型AI流程的评估,超越传统性能指标,引入任务成功所必需的社会动态因素。
原文摘要 · Abstract (English)
This paper presents an evaluation framework for agentic AI systems in mission-critical negotiation contexts, addressing the need for AI agents that can adapt to diverse human operators and stakeholders. Using Sotopia as a simulation testbed, we present two experiments that systematically evaluated how personality traits and AI agent characteristics influence LLM-simulated social negotiation outcomes--a capability essential for a variety of applications involving cross-team coordination and civil-military interactions. Experiment 1 employs causal discovery methods to measure how personality traits impact price bargaining negotiations, through which we found that Agreeableness and Extraversion significantly affect believability, goal achievement, and knowledge acquisition outcomes. Sociocognitive lexical measures extracted from team communications detected fine-grained differences in agents' empathic communication, moral foundations, and opinion patterns, providing actionable insights for agentic AI systems that must operate reliably in high-stakes operational scenarios. Experiment 2 evaluates human-AI job negotiations by manipulating both simulated human personality and AI system characteristics, specifically transparency, competence, adaptability, demonstrating how AI agent trustworthiness impact mission effectiveness. These findings establish a repeatable evaluation methodology for experimenting with AI agent reliability across diverse operator personalities and human-agent team dynamics, directly supporting operational requirements for reliable AI systems. Our work advances the evaluation of agentic AI workflows by moving beyond standard performance metrics to incorporate social dynamics essential for mission success in complex operations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。