用大模型知识蒸馏出小模型，零样本识别情绪更灵活。
Contrastive Distillation of Emotion Knowledge from LLMs for Zero-Shot Emotion Recognition
- 通过对比学习从大模型提取情感知识，无需人工标注。
- 在多个数据集上表现优异，接近GPT-4性能但体积小一万倍。
- 支持任意情绪类别、粒度和标签体系，适合边缘部署。
能够不依赖特定训练标签而处理多种情绪类别，对构建灵活的情绪识别系统至关重要。传统情绪识别模型需针对固定标签集训练，难以泛化。相比之下，大语言模型在跨标签空间的零样本情绪识别中表现出色，但其规模限制了在边缘设备的应用。本文提出一种对比蒸馏框架，将大模型中的丰富情感知识迁移到紧凑模型中，无需人类标注。利用GPT-4生成描述性情感标注，提供超越固定标签集的丰富监督信号。通过在共享嵌入空间中对齐文本与情感描述符，该方法实现对不同情绪类别、粒度和标签架构的零样本预测。蒸馏模型在多个数据集和标签空间上均有效，性能优于同规模强基线，接近GPT-4零样本表现，且体积超过10,000倍缩小。
原文摘要 · Abstract (English)
The ability to handle various emotion labels without dedicated training is crucial for building adaptable Emotion Recognition (ER) systems. Conventional ER models rely on training using fixed label sets and struggle to generalize beyond them. On the other hand, Large Language Models (LLMs) have shown strong zero-shot ER performance across diverse label spaces, but their scale limits their use on edge devices. In this work, we propose a contrastive distillation framework that transfers rich emotional knowledge from LLMs into a compact model without the use of human annotations. We use GPT-4 to generate descriptive emotion annotations, offering rich supervision beyond fixed label sets. By aligning text samples with emotion descriptors in a shared embedding space, our method enables zero-shot prediction on different emotion classes, granularity, and label schema. The distilled model is effective across multiple datasets and label spaces, outperforming strong baselines of similar size and approaching GPT-4's zero-shot performance, while being over 10,000 times smaller.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。