用异构图学习预测社交网络中用户情绪,融合多模态数据提升准确率。
A Heterogeneous Multimodal Graph Learning Framework for Recognizing User Emotions in Social Networks
- 基于异构图学习构建多模态用户情绪预测框架。
- 在多个数据集上显著优于传统手工特征方法,提升情感识别精度。
- 适合研究情感计算、社交网络分析与多模态深度学习的学者。
社交媒体平台的迅猛发展带来了海量多模态用户生成内容。理解用户情绪可为改善沟通与洞察人类行为提供重要价值。尽管情感计算领域已取得显著进展,但影响社交网络中用户情绪的多元因素仍研究不足。此外,现有深度学习方法在社交网络情绪预测方面仍较匮乏,难以充分利用丰富的多模态数据。本文提出一种基于异构图学习的个性化情绪预测新范式,并设计 HMG-Emo 框架,利用深度学习特征进行用户情绪识别。该框架包含动态上下文融合模块,可自适应整合社交媒体中的多模态信息。大量实验表明,HMG-Emo 效果显著,基于图神经网络的方法优于采用丰富手工特征的现有基线。据我们所知,HMG-Emo 是首个用于在线社交网络中个性化情绪预测的多模态深度学习方法,凸显了先进深度学习技术在情感计算未充分探索问题中的潜力。
原文摘要 · Abstract (English)
The rapid expansion of social media platforms has provided unprecedented access to massive amounts of multimodal user-generated content. Comprehending user emotions can provide valuable insights for improving communication and understanding of human behaviors. Despite significant advancements in Affective Computing, the diverse factors influencing user emotions in social networks remain relatively understudied. Moreover, there is a notable lack of deep learning-based methods for predicting user emotions in social networks, which could be addressed by leveraging the extensive multimodal data available. This work presents a novel formulation of personalized emotion prediction in social networks based on heterogeneous graph learning. Building upon this formulation, we design HMG-Emo, a Heterogeneous Multimodal Graph Learning Framework that utilizes deep learning-based features for user emotion recognition. Additionally, we include a dynamic context fusion module in HMG-Emo that is capable of adaptively integrating the different modalities in social media data. Through extensive experiments, we demonstrate the effectiveness of HMG-Emo and verify the superiority of adopting a graph neural network-based approach, which outperforms existing baselines that use rich hand-crafted features. To the best of our knowledge, HMG-Emo is the first multimodal and deep-learning-based approach to predict personalized emotions within online social networks. Our work highlights the significance of exploiting advanced deep learning techniques for less-explored problems in Affective Computing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。