提出分阶段蒸馏方法，让小型情感分析模型更精准、泛化能力更强。
Targeted Distillation for Sentiment Analysis
- 将知识与对齐分离，分两阶段进行模型压缩。
- 在12个数据集上验证，小模型性能显著提升。
- 适合需要轻量高效情感分析的工程应用。
本文探索面向情感分析的定向蒸馏方法，旨在构建紧凑且实用的模型，同时保持强大且可泛化的分析能力。为此，我们概念性地将蒸馏目标拆分为知识与对齐，并提出一种两阶段蒸馏框架。此外，我们引入SentiBench，一个覆盖12个数据集、涵盖多样任务的系统性情感分析基准。我们在该基准上评估了多种模型。实验结果表明，所提方法显著提升了紧凑模型在多样化情感分析任务上的表现，且其在未见任务上展现出强泛化能力，整体竞争力优于现有小型模型。
原文摘要 · Abstract (English)
This paper explores targeted distillation methods for sentiment analysis, aiming to build compact and practical models that preserve strong and generalizable sentiment analysis capabilities. To this end, we conceptually decouple the distillation target into knowledge and alignment and accordingly propose a two-stage distillation framework. Moreover, we introduce SentiBench, a comprehensive and systematic sentiment analysis benchmark that covers a diverse set of tasks across 12 datasets. We evaluate a wide range of models on this benchmark. Experimental results show that our approach substantially enhances the performance of compact models across diverse sentiment analysis tasks, and the resulting models demonstrate strong generalization to unseen tasks, showcasing robust competitiveness against existing small-scale models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。