给Transformer模型加了仿激素情绪模块,让生成文本更懂人类情感。
A Hormone-inspired Emotion Layer for Transformer language models (HELT)

- 用六种类激素值模拟人类情绪,通过专用注意力头动态计算。
- 在情感数据集上激素预测准确率超85%,情绪差异度超过0.85。
- 生成回复更符合情感语境,人评更显共情,适合聊天机器人优化。
大语言模型虽能生成语法正确、上下文相关的文本,但缺乏类人情感认知能力。现有NLP情感建模多依赖离散分类或简单情感分析,难以捕捉人类情绪的连续与多维特性。本文提出HormoneT5,一种通过生物启发的激素情绪模块增强Transformer模型的新架构。该模块通过六个独立注意力头分别计算类激素值,每个头具有正交初始化查询、温度缩放注意力及深层输出投影。这些激素值被转换为情感嵌入,调制编码器隐藏状态,实现情感适配的响应生成。我们设计了多目标训练框架,包含序列到序列损失、带间隔惩罚的激素预测损失及多样性正则化,防止注意力坍塌。在自建的情感标注数据集上,HormoneT5在0.15容忍阈值下每种激素预测准确率超85%,六种激素在不同情绪基调间的区分度均超过0.85。人工评估显示,其生成回复在情感恰当性和共情质量上显著优于基线T5(p < 0.01)。本工作为生物启发的情感计算和情感智能对话系统开辟新路径。
原文摘要 · Abstract (English)
Large Language Models have demonstrated remarkable capabilities in generating contextually relevant and grammatically correct text. However, they fundamentally lack the ability to process and respond to emotional context in a manner analogous to human emotional cognition. Current approaches to emotion modeling in NLP systems rely primarily on discrete emotion classification or simplistic sentiment analysis, which fail to capture the continuous, multi-dimensional nature of human emotional states. In this paper, we introduce HormoneT5, a novel architecture that augments transformer language models with a biologically-inspired Hormone Emotion Block that simulates the human endocrine system's role in emotional processing. Our approach computes six continuous hormone-like values through specialized per-hormone attention heads, each with orthogonally initialized learnable queries, temperature-scaled attention mechanisms, and deep output projections. These hormone values are then transformed into an emotional embedding that modulates the encoder hidden states, enabling emotionally-appropriate response generation. We propose a multi-objective training framework combining sequence-to-sequence loss, hormone prediction loss with margin penalties, and diversity regularization to prevent attention collapse. Experimental results on our curated emotion-labeled dataset demonstrate that HormoneT5 achieves 85%+ per-hormone accuracy within a 0.15 tolerance threshold, with hormone differentiation ranges exceeding 0.85 across all six hormones between contrasting emotional tones. Human evaluation studies show significant preference (p < 0.01) for HormoneT5-generated responses in terms of emotional appropriateness and empathetic quality compared to baseline T5 outputs. Our work opens new directions for biologically-grounded affective computing and emotionally intelligent conversational agents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。