用自动构造指令和难度筛选，让小模型高效学大模型的语义分析能力。
Comprehensive and Efficient Distillation for Lightweight Sentiment Analysis Models
- 自动生成多样化指令，弥补人工指令不足
- 仅用10%数据即达基线性能，3B小模型媲美20倍大的教师模型
- 适合资源有限但需高性能情感分析的场景
近期研究利用知识蒸馏构建轻量级情感分析模型，依赖人工编写指令和大规模用户文本。然而仍面临两大挑战：(1) 人工指令数量与多样性有限，难以全面覆盖蒸馏知识；(2) 大规模用户文本带来高计算成本，影响实用性。为此，我们提出CompEffDist框架，包含基于属性的自动指令生成与基于难度的数据过滤模块，分别应对上述问题。在多个模型系列（Llama-3、Qwen-3、Gemma-3）上应用该方法，使3B参数的学生模型在多数任务上达到20倍于其规模的教师模型性能。同时，本方法在数据效率上显著优于基线，仅需10%的数据即可实现相同性能。
原文摘要 · Abstract (English)
Recent efforts leverage knowledge distillation techniques to develop lightweight and practical sentiment analysis models. These methods are grounded in human-written instructions and large-scale user texts. Despite the promising results, two key challenges remain: (1) manually written instructions are limited in diversity and quantity, making them insufficient to ensure comprehensive coverage of distilled knowledge; (2) large-scale user texts incur high computational cost, hindering the practicality of these methods. To this end, we introduce CompEffDist, a comprehensive and efficient distillation framework for sentiment analysis. Our framework consists of two key modules: attribute-based automatic instruction construction and difficulty-based data filtering, which correspondingly tackle the aforementioned challenges. Applying our method across multiple model series (Llama-3, Qwen-3, and Gemma-3), we enable 3B student models to match the performance of 20x larger teacher models on most tasks. In addition, our approach greatly outperforms baseline methods in data efficiency, attaining the same performance level with only 10% of the data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。