小模型集合通过可信度与置信度加权,情绪识别效果超越大模型。
Confidence-Credibility Aware Weighted Ensembles of Small LLMs Outperform Large LLMs in Emotion Detection
- 用不同架构的小模型组合,保留错误多样性,避免参数趋同。
- 在DAIR-AI数据集上达93.5%宏F1,超过多个70亿参数的大模型。
- 适合追求高精度、低资源的专用NLP任务,如情感分析。
本文提出一种基于信心加权与可信度感知的集成框架,用于文本情绪检测,受康多塞陪审团定理启发。不同于传统同构模型集成,该方法融合五种不同架构的小型Transformer语言模型(sLLMs):BERT、RoBERTa、DistilBERT、DeBERTa和ELECTRA,均针对情绪分类任务全量微调。为保持错误多样性,最小化参数收敛,同时利用各模型独特偏差。采用双权重投票机制,结合全局可信度(验证集F1分数)与局部置信度(实例级概率),动态调整模型贡献。在DAIR-AI数据集上的实验表明,该可信度-信心集成达到93.5%宏F1,超越现有最佳基准,并显著优于经过任务特定低秩适应(LoRA)后的大型模型(如Falcon、Mistral、Qwen、Phi)。总参数仅595M,证明小型微调模型集成在情绪检测等专用任务中,比高达70亿参数的模型更具参数效率与鲁棒性。
原文摘要 · Abstract (English)
This paper introduces a confidence-weighted, credibility-aware ensemble framework for text-based emotion detection, inspired by Condorcet's Jury Theorem (CJT). Unlike conventional ensembles that often rely on homogeneous architectures, our approach combines architecturally diverse small transformer-based large language models (sLLMs) - BERT, RoBERTa, DistilBERT, DeBERTa, and ELECTRA, each fully fine-tuned for emotion classification. To preserve error diversity, we minimize parameter convergence while taking advantage of the unique biases of each model. A dual-weighted voting mechanism integrates both global credibility (validation F1 score) and local confidence (instance-level probability) to dynamically weight model contributions. Experiments on the DAIR-AI dataset demonstrate that our credibility-confidence ensemble achieves a macro F1 score of 93.5 percent, surpassing state-of-the-art benchmarks and significantly outperforming large-scale LLMs, including Falcon, Mistral, Qwen, and Phi, even after task-specific Low-Rank Adaptation (LoRA). With only 595M parameters in total, our small LLMs ensemble proves more parameter-efficient and robust than models up to 7B parameters, establishing that carefully designed ensembles of small, fine-tuned models can outperform much larger LLMs in specialized natural language processing (NLP) tasks such as emotion detection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。