融合情感与话题标签的注意力模型,提升多模态帖子热度预测精度
Sentiment and Hashtag-aware Attentive Deep Neural Network for Multimodal Post Popularity Prediction
- 通过人脸视觉人口统计与话题标签情感分析,挖掘多模态深层特征
- 在两个真实数据集上超越现有方法,显著提升热度预测准确率
- 适合关注社交媒体内容推荐与用户情绪感知的研究者
社交媒体用户通过包含文本、图像等多模态表达的内容分享观点与经历,导致平台上多模态内容激增。然而,准确预测这类帖子的热度仍具挑战。现有方法多聚焦内容本身,忽视了视觉人口统计、话题标签情感以及三者间复杂关系带来的信息价值。为此,本文提出NARRATOR模型,从图像中提取人脸视觉人口统计信息,通过话题标签识别情感倾向,并引入话题引导的注意力机制,引导模型聚焦于文本与视觉模态中与目标受众兴趣和社交语境相关的特征。实验表明,NARRATOR在两个真实数据集上显著优于现有方法。消融实验验证了视觉人口统计、话题情感分析及话题引导注意力机制的有效性,提升了帖子的受众相关性、情感共鸣与审美吸引力。
原文摘要 · Abstract (English)
Social media users articulate their opinions on a broad spectrum of subjects and share their experiences through posts comprising multiple modes of expression, leading to a notable surge in such multimodal content on social media platforms. Nonetheless, accurately forecasting the popularity of these posts presents a considerable challenge. Prevailing methodologies primarily center on the content itself, thereby overlooking the wealth of information encapsulated within alternative modalities such as visual demographics, sentiments conveyed through hashtags and adequately modeling the intricate relationships among hashtags, texts, and accompanying images. This oversight limits the ability to capture emotional connection and audience relevance, significantly influencing post popularity. To address these limitations, we propose a seNtiment and hAshtag-aware attentive deep neuRal netwoRk for multimodAl posT pOpularity pRediction, herein referred to as NARRATOR that extracts visual demographics from faces appearing in images and discerns sentiment from hashtag usage, providing a more comprehensive understanding of the factors influencing post popularity Moreover, we introduce a hashtag-guided attention mechanism that leverages hashtags as navigational cues, guiding the models focus toward the most pertinent features of textual and visual modalities, thus aligning with target audience interests and broader social media context. Experimental results demonstrate that NARRATOR outperforms existing methods by a significant margin on two real-world datasets. Furthermore, ablation studies underscore the efficacy of integrating visual demographics, sentiment analysis of hashtags, and hashtag-guided attention mechanisms in enhancing the performance of post popularity prediction, thereby facilitating increased audience relevance, emotional engagement, and aesthetic appeal.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。