用大模型生成10万条带上下文的情绪数据,让小模型更准地识别细粒度情绪。
Emo Pillars: Knowledge Distillation to Support Fine-Grained Context-Aware and Context-Less Emotion Classification
- 用Mistral-7b生成故事化语句,提升情绪样本多样性
- 构建10万条带上下文+30万条无上下文数据,覆盖28种情绪
- 轻量模型在多个数据集上达最优,适合情绪分析新手和研究者
情感分析数据集常缺乏表达观点的上下文,且情绪类别有限。大型语言模型如GPT-4存在过度预测情绪且资源消耗大的问题。本文设计基于LLM的数据合成流程,利用Mistral-7b生成训练样本,用于微调轻量级BERT类编码器。通过将生成过程锚定于叙事语料库,产生以角色为中心、具有独特上下文的非重复语句,覆盖28种情绪类别。在450 GPU小时内完成70万次推理,构建包含10万条带上下文和30万条无上下文示例的数据集。使用该数据集微调预训练编码器,得到Emo Pillars系列模型。实验表明,这些模型在GoEmotions、ISEAR、IEMOCAP和EmoContext等任务中表现优异,尤其在前三个数据集达到当前最佳性能。通过统计分析与人工评估验证了生成样本在语义多样性和上下文个性化方面的有效性(中性情绪改善较弱),并指出需优化对未定义标签的处理。
原文摘要 · Abstract (English)
Most datasets for sentiment analysis lack context in which an opinion was expressed, often crucial for emotion understanding, and are mainly limited by a few emotion categories. Foundation large language models (LLMs) like GPT-4 suffer from over-predicting emotions and are too resource-intensive. We design an LLM-based data synthesis pipeline and leverage a large model, Mistral-7b, for the generation of training examples for more accessible, lightweight BERT-type encoder models. We focus on enlarging the semantic diversity of examples and propose grounding the generation into a corpus of narratives to produce non-repetitive story-character-centered utterances with unique contexts over 28 emotion classes. By running 700K inferences in 450 GPU hours, we contribute with the dataset of 100K contextual and also 300K context-less examples to cover both scenarios. We use it for fine-tuning pre-trained encoders, which results in several Emo Pillars models. We show that Emo Pillars models are highly adaptive to new domains when tuned to specific tasks such as GoEmotions, ISEAR, IEMOCAP, and EmoContext, reaching the SOTA performance on the first three. We also validate our dataset, conducting statistical analysis and human evaluation, and confirm the success of our measures in utterance diversification (although less for the neutral class) and context personalization, while pointing out the need for improved handling of out-of-taxonomy labels within the pipeline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。