小模型实现强情感推理，参数少却性能超大模型。
TinyEmo: Scaling down Emotional Reasoning via Metric Projection
- 用合成数据+度量投影器，让小模型高效做情感分析。
- 700M模型在情感分类上超越70亿参数的主流模型。
- 无需额外训练即可检测模型偏见，适合可解释性研究者。
本文提出TinyEmo，一套用于情感推理与分类的小型多模态语言模型。方法包括：(1) 为预训练与微调阶段构建合成情感指令数据集；(2) 引入度量投影器（Metric Projector），将分类任务从语言模型中分离，提升训练与推理效率；(3) 使用多模态大语言模型（MM-LLM）进行情感推理；(4) 构建半自动化偏见检测框架。TinyEmo在显著减少参数量的前提下完成情感分类与推理任务。其最小模型仅700M参数，性能却优于基于通用MM-LLM（超过70亿参数）的先进模型。此外，度量投影器支持无需额外训练的可解释性分析与间接偏见检测，有助于理解并改进AI系统。代码、模型与数据集已开源于https://github.com/ggcr/TinyEmo。
原文摘要 · Abstract (English)
This paper introduces TinyEmo, a family of small multi-modal language models for emotional reasoning and classification. Our approach features: (1) a synthetic emotional instruct dataset for both pre-training and fine-tuning stages, (2) a Metric Projector that delegates classification from the language model allowing for more efficient training and inference, (3) a multi-modal large language model (MM-LLM) for emotional reasoning, and (4) a semi-automated framework for bias detection. TinyEmo is able to perform emotion classification and emotional reasoning, all while using substantially fewer parameters than comparable models. This efficiency allows us to freely incorporate more diverse emotional datasets, enabling strong performance on classification tasks, with our smallest model (700M parameters) outperforming larger state-of-the-art models based on general-purpose MM-LLMs with over 7B parameters. Additionally, the Metric Projector allows for interpretability and indirect bias detection in large models without additional training, offering an approach to understand and improve AI systems. We release code, models, and dataset at https://github.com/ggcr/TinyEmo
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。