小模型统一六项情感任务,从感知到共情实现端到端情感智能。
Nano-EmoX: Unifying Multimodal Emotional Intelligence from Perception to Empathy
- 构建三层认知层级:感知、理解、互动,统一情感建模框架。
- 2.2B参数模型在6个任务上达顶尖表现,跨任务迁移性强。
- 适合追求高效情感建模的开发者和研究者使用。
情感多模态语言模型的发展长期受限于低层感知与高层交互之间的鸿沟,导致情感能力碎片化且泛化性差。为此,我们提出一种类脑三层次架构,按认知深度组织情感任务——感知、理解、互动,并为情感建模提供统一概念基础。基于此,我们推出小型多任务多模态语言模型 Nano-EmoX 及 P2E(Perception-to-Empathy)课程式训练框架。Nano-EmoX 集成一系列全模态编码器,包括增强型面部编码器与融合编码器,以捕捉关键多模态情感线索,提升跨任务迁移能力。其输出通过异构适配器投影至统一语言空间,使轻量语言模型可处理多样情感任务。同时,P2E 通过逐步对齐快速感知与思维链驱动的共情,持续培养情感智能。据我们所知,Nano-EmoX 是首个统一六个核心情感任务(覆盖三层次)的小型模型(2.2B),在多个基准测试中达到或接近顶尖性能,展现出卓越效率与泛化能力。代码已开源:https://github.com/waHAHJIAHAO/Nano-EmoX。
原文摘要 · Abstract (English)
The development of affective multimodal language models (MLMs) has long been constrained by a gap between low-level perception and high-level interaction, leading to fragmented affective capabilities and limited generalization. To bridge this gap, we propose a cognitively inspired three-level hierarchy that organizes affective tasks according to their cognitive depth-perception, understanding, and interaction-and provides a unified conceptual foundation for advancing affective modeling. Guided by this hierarchy, we introduce Nano-EmoX, a small-scale multitask MLM, and P2E (Perception-to-Empathy), a curriculum-based training framework. Nano-EmoX integrates a suite of omni-modal encoders, including an enhanced facial encoder and a fusion encoder, to capture key multimodal affective cues and improve cross-task transferability. The outputs are projected into a unified language space via heterogeneous adapters, empowering a lightweight language model to tackle diverse affective tasks. Concurrently, P2E progressively cultivates emotional intelligence by aligning rapid perception with chain-of-thought-driven empathy. To the best of our knowledge, Nano-EmoX is the first compact MLM (2.2B) to unify six core affective tasks across all three hierarchy levels, achieving state-of-the-art or highly competitive performance across multiple benchmarks, demonstrating excellent efficiency and generalization. The code is available at https://github.com/waHAHJIAHAO/Nano-EmoX.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。