arXiv:2512.10963cs.IRcs.AI2025-12被引 2

用多模态情感与意图分析,让AI内容推荐更懂用户情绪。

Emotion-Driven Personalized Recommendation for AI-Generated Content Using Multi-Modal Sentiment and Intent Analysis

  • 融合视觉、语音和文本三模态,用注意力机制提取情感意图特征。
  • 相比基线模型,F1-score提升4.3%,交叉熵损失降低12.3%。
  • 适合打造有共情能力的下一代AI内容推荐系统,尤其关注用户体验。

随着AI生成内容(AIGC)在音乐、视频、文学等领域的快速发展,情感感知型推荐系统的重要性日益凸显。传统推荐系统主要依赖点击、观看或评分等行为数据,忽视了用户在内容交互过程中的实时情感与意图状态。为此,本文提出基于BERT的跨模态变换器与注意力融合的多模态情感与意图识别模型(MMEI),集成于云原生个性化AIGC推荐框架中。该系统通过预训练编码器ViT、Wav2Vec2和BERT分别处理面部表情、语音语调和评论/话语文本,再经注意力融合模块学习情感-意图表征,并通过上下文匹配层实现个性化推荐。在基准情感数据集(AIGC-INT、MELD、CMU-MOSEI)及AIGC交互数据集上的实验表明,所提MMEI模型相较最优融合式Transformer基线,F1-score提升4.3%,交叉熵损失降低12.3%。此外,用户在线评估显示,情感驱动推荐使互动时长增加15.2%,满意度提升11.8%,验证了模型在对齐用户情感与意图状态方面的有效性。本研究揭示了跨模态情感智能在下一代AIGC生态中的潜力,推动实现自适应、共情化与情境感知的推荐体验。

原文摘要 · Abstract (English)

With the rapid growth of AI-generated content (AIGC) across domains such as music, video, and literature, the demand for emotionally aware recommendation systems has become increasingly important. Traditional recommender systems primarily rely on user behavioral data such as clicks, views, or ratings, while neglecting users' real-time emotional and intentional states during content interaction. To address this limitation, this study proposes a Multi-Modal Emotion and Intent Recognition Model (MMEI) based on a BERT-based Cross-Modal Transformer with Attention-Based Fusion, integrated into a cloud-native personalized AIGC recommendation framework. The proposed system jointly processes visual (facial expression), auditory (speech tone), and textual (comments or utterances) modalities through pretrained encoders ViT, Wav2Vec2, and BERT, followed by an attention-based fusion module to learn emotion-intent representations. These embeddings are then used to drive personalized content recommendations through a contextual matching layer. Experiments conducted on benchmark emotion datasets (AIGC-INT, MELD, and CMU-MOSEI) and an AIGC interaction dataset demonstrate that the proposed MMEI model achieves a 4.3% improvement in F1-score and a 12.3% reduction in cross-entropy loss compared to the best fusion-based transformer baseline. Furthermore, user-level online evaluations reveal that emotion-driven recommendations increase engagement time by 15.2% and enhance satisfaction scores by 11.8%, confirming the model's effectiveness in aligning AI-generated content with users' affective and intentional states. This work highlights the potential of cross-modal emotional intelligence for next-generation AIGC ecosystems, enabling adaptive, empathetic, and context-aware recommendation experiences.

情感分析多模态推荐系统AIGC

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。