arXiv:2410.17632cs.CLcs.AI2024-10被引 23

给大模型做性格测试,量化其语言个性特征。

LMLPA: Language Model Linguistic Personality Assessment

  • 用自适应版大五人格问卷+开放文本回答评估模型性格。
  • 实验证明大模型有可量化的独特语言个性。
  • 适合关注AI人格、人机交互的研究者使用。

大型语言模型(LLMs)在日常应用和研究中日益普及,尤其在对话交互中表现突出。与人类对话类似,人机对话的体验取决于对话双方的性格特质。然而,当前尚缺乏有效手段衡量大模型的语言性格。本文提出语言模型语言个性评估系统(LMLPA),通过定量分析大模型语言输出中的个性特征来理解其生成能力。该系统基于大五人格量表(Big Five Inventory)并结合过往语言性格测量研究,适配大模型特性,采用开放式问题设计以减少选项顺序偏差,生成文本回答。再由人工智能评分器将模糊文本信息转化为明确的个性数值指标。经主成分分析与信度验证,结果表明大模型具备可被有效量化的独特个性特征。本研究推动人机交互与以人为中心的人工智能发展,为未来优化AI性格评估提供可靠框架,并拓展其在教育、制造等领域的应用前景。

原文摘要 · Abstract (English)

Large Language Models (LLMs) are increasingly used in everyday life and research. One of the most common use cases is conversational interactions, enabled by the language generation capabilities of LLMs. Just as between two humans, a conversation between an LLM-powered entity and a human depends on the personality of the conversants. However, measuring the personality of a given LLM is currently a challenge. This paper introduces the Language Model Linguistic Personality Assessment (LMLPA), a system designed to evaluate the linguistic personalities of LLMs. Our system helps to understand LLMs' language generation capabilities by quantitatively assessing the distinct personality traits reflected in their linguistic outputs. Unlike traditional human-centric psychometrics, the LMLPA adapts a personality assessment questionnaire, specifically the Big Five Inventory, to align with the operational capabilities of LLMs, and also incorporates the findings from previous language-based personality measurement literature. To mitigate sensitivity to the order of options, our questionnaire is designed to be open-ended, resulting in textual answers. Thus, the AI rater is needed to transform ambiguous personality information from text responses into clear numerical indicators of personality traits. Utilising Principal Component Analysis and reliability validations, our findings demonstrate that LLMs possess distinct personality traits that can be effectively quantified by the LMLPA. This research contributes to Human-Computer Interaction and Human-Centered AI, providing a robust framework for future studies to refine AI personality assessments and expand their applications in multiple areas, including education and manufacturing.

大模型性格人机交互语言评估

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。