发现大模型对话中状态影响远大于性格，但模型却无视状态变化。
Beyond Fixed Psychological Personas: State Beats Trait, but Language Models are State-Blind
- 基于心理状态-特质理论分解用户行为，发现74%差异来自即时状态。
- 大模型仅关注用户性格，对同一人不同情境输出相似回复。
- 奖励模型对状态反应不一，可能引发对齐偏差，适合对话研究者参考。
用户与语言模型的交互受静态人格特征（特质）和具体情境（状态）双重影响。然而现有角色数据集（如PersonaChat、PANDORA等）仅捕捉特质，忽略状态作用。本文提出Chameleon数据集，包含1,667名Reddit用户的5,001个情境化心理画像，覆盖多场景。基于此，我们得出三个关键结论：第一，依据潜在状态-特质理论分解方差，发现74%的个体差异源于状态（在个人内部），仅26%来自特质（跨个体）；第二，大语言模型呈现“状态盲”现象，只依赖特质信息，对同一用户在不同情境下生成几乎相同的回应；第三，奖励模型虽能感知用户状态，但反应不一致——不同模型对同一用户在相同情境中表现出相反的偏好或惩罚。我们公开Chameleon数据集，以支持情感计算、个性化对话及强化学习人类反馈对齐的研究。
原文摘要 · Abstract (English)
User interactions with language models vary due to static properties of the user (trait) and the specific context of the interaction (state). However, existing persona datasets (like PersonaChat, PANDORA etc.) capture only trait, and ignore the impact of state. We introduce Chameleon, a dataset of 5,001 contextual psychological profiles from 1,667 Reddit users, each measured across multiple contexts. Using the Chameleon dataset, we present three key findings. First, inspired by Latent State-Trait theory, we decompose variance and find that 74% is within-person(state) while only 26% is between-person (trait). Second, we find that LLMs are state-blind: they focus on trait only, and produce similar responses regardless of state. Third, we find that reward models react to user state, but inconsistently: different models favor or penalize the same users in opposite directions. We release Chameleon to support research on affective computing, personalized dialogue, and RLHF alignment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。