针对尼泊尔语表情包的仇恨言论与情感分析，提出跨模态融合新方法。
MEME-Fusion@CHiPSAL 2026: Multimodal Ablation Study of Hate Detection and Sentiment Analysis on Nepali Memes
- 用视觉与文本双模态结合，通过动态加权机制融合特征
- 跨模态模型在仇恨检测上提升5.9%的宏平均F1分数
- 发现英文视觉模型对天城文无效，小样本下集成方法会失效
天城文字母社交平台表情包中的仇恨言论检测面临多重挑战：多模态内容结构、语言复杂性以及低资源环境下数据极度稀缺。本文针对CHiPSAL 2026共享任务，解决子任务A（二分类仇恨言论检测）和子任务B（三分类情感分析：正面、中性、负面）。提出一种混合跨模态注意力融合架构，将CLIP（ViT-B/32）用于视觉编码，BGE-M3实现多语言文本表征，通过4头自注意力与可学习门控网络连接，按样本动态调整模态权重。在八种模型配置下的系统评估表明，显式跨模态推理使子任务A的F1-macro相比纯文本基线提升5.9%；同时发现两个意外但关键的结果：以英语为中心的视觉模型在天城文上表现接近随机，标准集成方法在数据稀缺（每折约850条）时因相关过拟合而严重退化。代码已开源至https://github.com/Tri-Yantra-Technologies/MEME-Fusion/
原文摘要 · Abstract (English)
Hate speech detection in Devanagari-scripted social media memes presents compounded challenges: multimodal content structure, script-specific linguistic complexity, and extreme data scarcity in low-resource settings. This paper presents our system for the CHiPSAL 2026 shared task, addressing both Subtask A (binary hate speech detection) and Subtask B (three-class sentiment classification: positive, neutral, negative). We propose a hybrid cross-modal attention fusion architecture that combines CLIP (ViT-B/32) for visual encoding with BGE-M3 for multilingual text representation, connected through 4-head self-attention and a learnable gating network that dynamically weights modality contributions on a per-sample basis. Systematic evaluation across eight model configurations demonstrates that explicit cross-modal reasoning achieves a 5.9% F1-macro improvement over text-only baselines on Subtask A, while uncovering two unexpected but critical findings: English-centric vision models exhibit near-random performance on Devanagari script, and standard ensemble methods catastrophically degrade under data scarcity (N nearly equal to 850 per fold) due to correlated overfitting. The code can be accessed at https://github.com/Tri-Yantra-Technologies/MEME-Fusion/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。