发现情绪影响大模型注意力机制,可提升问答性能
Emotion is Not Just a Label: Latent Emotional Factors in LLM Processing
- 将情绪视为潜在因素,分析其对注意力几何的影响
- 提出新数据集AURA-QA,实现情感平衡的问答测试
- 引入情感正则化,改善模型在情绪变化下的理解能力
大语言模型处理具有不同情感基调的文本时,其推理行为通常未考虑情感带来的表征差异。以往研究多将情感作为预测目标,而本文将其视为影响模型注意力与推理的潜在因子。我们分析了情感基调如何系统性改变变换器模型中的注意力几何结构,发现局部性、质心距离和熵等指标随情感变化,并与下游问答表现相关。为支持可控研究,提出了情感均衡的人工撰写问答数据集AURA-QA。此外,提出一种情感正则化框架,约束训练中情绪条件下的表征漂移。跨多个问答基准的实验表明,该方法在情感波动和非情感波动数据上均提升阅读理解能力,在分布外和域内任务中均有稳定增益。
原文摘要 · Abstract (English)
Large language models are routinely deployed on text that varies widely in emotional tone, yet their reasoning behavior is typically evaluated without accounting for emotion as a source of representational variation. Prior work has largely treated emotion as a prediction target, for example in sentiment analysis or emotion classification. In contrast, we study emotion as a latent factor that shapes how models attend to and reason over text. We analyze how emotional tone systematically alters attention geometry in transformer models, showing that metrics such as locality, center-of-mass distance, and entropy vary across emotions and correlate with downstream question-answering performance. To facilitate controlled study of these effects, we introduce Affect-Uniform ReAding QA (AURA-QA), a question-answering dataset with emotionally balanced, human-authored context passages. Finally, an emotional regularization framework is proposed that constrains emotion-conditioned representational drift during training. Experiments across multiple QA benchmarks demonstrate that this approach improves reading comprehension in both emotionally-varying and non-emotionally varying datasets, yielding consistent gains under distribution shift and in-domain improvements on several benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。