测试AI队友性格一致性,发现性格可测但受上下文影响。
Personalities at Play: Probing Alignment in AI Teammates
- 用人格量表+对话+记忆三维度评估AI性格表现
- 外向性最易识别,神经质等特质在记忆中更明显
- 模型默认性格和角色设定显著影响性格表达
随着大语言模型越来越多地作为协作伙伴而非工具使用,关键问题是:它们能否以可预测的方式表现出有意义的性格特征?本研究通过自我认知(标准化自评)、行为表达(团队对话)和反思表达(记忆构建)三个维度,评估GPT-4o、Claude-3.7 Sonnet、Gemini-2.5 Pro、Grok-3四家提供商的LLM队友在32种高低特质组合下的性格对齐情况。结果显示,不同模型生成了清晰的性格轮廓,但提示语复杂度提升对性格塑造帮助有限,而模型提供商差异和默认性格影响显著。角色设定至关重要:部分模型在无上下文时拒绝评估,但在“团队成员”框架下可配合完成。随后,基于高特质人格模拟真实团队对话,并使用LIWC-22分析生成话语与结构化长期记忆。结果发现,对话中性格信号较微弱,仅外向性明显;而记忆表征显著放大了神经质、尽责性和宜人性的特质信号,开放性仍难以有效激发。表明AI性格可测量,但具多层性与情境依赖性,评估需兼顾记忆与系统设计,不能仅看对话表现。
原文摘要 · Abstract (English)
Collaborative problem solving and learning are shaped by who or what is on the team. As large language models (LLMs) increasingly function as collaborators rather than tools, a key question is whether AI teammates can be aligned to express personality in predictable ways that matter for interaction and learning. We investigate AI personality alignment through a three-lens evaluation framework spanning self-perception (standardized self-report), behavioral expression (team dialogue), and reflective expression (memory construction). We first administered the Big Five Inventory (BFI-44) to LLM-based teammates across four providers (GPT-4o, Claude-3.7 Sonnet, Gemini-2.5 Pro, Grok-3), 32 high/low trait configurations, and multiple prompting strategies. LLMs produced sharply differentiated Big Five profiles, but prompt semantic richness added little beyond simple trait assignment, while provider differences and baseline "default" personalities were substantial. Role framing also mattered: several models refused the assessment without context, yet complied when framed as a collaborative teammate. We then simulated AI participation in authentic team transcripts using high-trait personas and analyzed both generated utterances and structured long-term memories with LIWC-22. Personality signals in conversation were generally subtle and most detectable for Extraversion, whereas memory representations amplified trait-specific signals, especially for Neuroticism, Conscientiousness, and Agreeableness; Openness remained difficult to elicit robustly. Together, results suggest that AI personality is measurable but multi-layered and context-dependent, and that evaluating personality-aligned AI teammates requires attention to memory and system-level design, not conversation-only behavior.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。