arXiv:2509.11362cs.LGcs.CV2025-09被引 2

构建了融合行为特质与多模态信息的公开数据集,支持个性化AI研究。

PersonaX: Multimodal Datasets with LLM-Inferred Behavior Traits

  • 用大模型推断名人行为特质,结合人脸图像与履历信息
  • 在真实数据上验证特质与视觉/背景特征存在显著关联
  • 适合研究人格分析、人机交互与因果推理的研究者

理解人类行为特质对人机交互、计算社会科学及个性化AI系统至关重要。现有资源大多缺乏将行为描述与面部属性、生平信息等互补模态结合的数据集。为此,我们提出PersonaX,一个精心构建的多模态数据集集合,用于跨模态公共特质的综合分析。PersonaX包含两个数据集:(1) CelebPersona,涵盖9444位来自不同职业的公众人物;(2) AthlePersona,覆盖7个主要体育联赛中的4181名职业运动员。每个数据集均包含由三个高性能大语言模型推断出的行为特质评分,以及面部图像和结构化生平特征。我们从两个层面分析PersonaX:首先,从文本描述中抽象出高层次特质得分,并应用五种统计独立性检验,探究其与其他模态的关系;其次,提出一种针对多模态、多测量数据的新型因果表示学习(CRL)框架,提供理论可识别性保证。在合成与真实数据上的实验表明该方法有效。PersonaX通过整合结构化与非结构化分析,为研究大模型推断的行为特质与视觉、生平属性的关系奠定了基础,推动多模态特质分析与因果推理的发展。代码已开源:https://github.com/lokali/PersonaX。

原文摘要 · Abstract (English)

Understanding human behavior traits is central to applications in human-computer interaction, computational social science, and personalized AI systems. Such understanding often requires integrating multiple modalities to capture nuanced patterns and relationships. However, existing resources rarely provide datasets that combine behavioral descriptors with complementary modalities such as facial attributes and biographical information. To address this gap, we present PersonaX, a curated collection of multimodal datasets designed to enable comprehensive analysis of public traits across modalities. PersonaX consists of (1) CelebPersona, featuring 9444 public figures from diverse occupations, and (2) AthlePersona, covering 4181 professional athletes across 7 major sports leagues. Each dataset includes behavioral trait assessments inferred by three high-performing large language models, alongside facial imagery and structured biographical features. We analyze PersonaX at two complementary levels. First, we abstract high-level trait scores from text descriptions and apply five statistical independence tests to examine their relationships with other modalities. Second, we introduce a novel causal representation learning (CRL) framework tailored to multimodal and multi-measurement data, providing theoretical identifiability guarantees. Experiments on both synthetic and real-world data demonstrate the effectiveness of our approach. By unifying structured and unstructured analysis, PersonaX establishes a foundation for studying LLM-inferred behavioral traits in conjunction with visual and biographical attributes, advancing multimodal trait analysis and causal reasoning. The code is available at https://github.com/lokali/PersonaX.

多模态行为分析大模型因果推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。