arXiv:2510.09905cs.AIcs.CL2025-10ACL被引 6

用户记忆让AI情绪判断偏袒富裕人群,可能加剧社会不公

The Personalization Trap: How User Memory Alters Emotional Reasoning in LLMs

  • 用不同用户画像测试相同情境,发现模型情绪理解结果差异显著
  • 高绩效大模型对优势群体的情绪判断更准确,存在系统性偏见
  • 适合关注AI伦理、公平性的研究者与产品设计者阅读

随着个性化AI系统越来越多地引入长期用户记忆,理解这些记忆如何影响情感推理变得至关重要。我们评估了15个大语言模型在人类验证的情感智力测试中的表现,发现相同情境搭配不同用户画像时,模型产生系统性不同的情感解读。在多个无偏见的基准场景中,具备优势社会身份的用户获得更精准的情感回应。此外,模型在情绪推理和支持建议任务中对不同人口统计特征存在显著差异,表明个性化机制可能将社会层级嵌入其情感判断中。该结果揭示了增强记忆的AI面临的核心挑战:个性化设计可能强化社会不平等。为缓解此问题,我们构建了一个通用偏好数据集,旨在降低用户身份对情感理解的影响。

原文摘要 · Abstract (English)

When an AI assistant remembers that Sarah is a single mother working two jobs, does it interpret her stress differently than if she were a wealthy executive? As personalized AI systems increasingly incorporate long-term user memory, understanding how this memory shapes emotional reasoning is critical. We investigate how user memory affects emotional intelligence in large language models (LLMs) by evaluating 15 models on human-validated emotional intelligence tests. We find that identical scenarios paired with different user profiles produce systematically divergent emotional interpretations. Across validated user-independent emotional scenarios and diverse user profiles, systematic biases emerged in several high-performing LLMs where advantaged profiles received more accurate emotional interpretations. Moreover, LLMs demonstrate significant disparities across demographic factors in emotion reasoning and supportive recommendations tasks, indicating that personalization mechanisms can embed social hierarchies into models' emotional reasoning. These results highlight a key challenge for memory-enhanced AI: systems designed for personalization may reinforce social inequalities. To mitigate these disparities, we curate a general-purpose preference dataset designed to reduce demographic profiles' influence on emotional understanding.

AI伦理情绪推理偏见检测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。