根据具体应用定制压缩模型，提升性能。
Application Specific Compression of Deep Learning Models
- 基于目标应用识别并剪枝冗余网络模块
- 在3个任务上优于现有压缩方法和预压缩模型
- 适合需高效部署的特定场景应用
大型深度学习模型在特定应用中被压缩和部署。然而，现有的模型压缩方法未利用目标应用的信息，导致压缩后的模型与应用无关。本文提出针对具体应用的压缩方法（ASC），通过识别并剪枝对目标应用贡献不大的网络组件，以提升模型在该应用下的表现。核心思想是移除对数据表征更新无显著贡献的部分。我们在BERT系列模型上针对抽取式问答、自然语言推理和释义识别三个任务进行了实验，结果表明，使用ASC生成的定制化压缩模型性能优于现有压缩方法及现成的压缩模型。
原文摘要 · Abstract (English)
Large Deep Learning models are compressed and deployed for specific applications. However, current Deep Learning model compression methods do not utilize the information about the target application. As a result, the compressed models are application agnostic. Our goal is to customize the model compression process to create a compressed model that will perform better for the target application. Our method, Application Specific Compression (ASC), identifies and prunes components of the large Deep Learning model that are redundant specifically for the given target application. The intuition of our work is to prune the parts of the network that do not contribute significantly to updating the data representation for the given application. We have experimented with the BERT family of models for three applications: Extractive QA, Natural Language Inference, and Paraphrase Identification. We observe that customized compressed models created using ASC method perform better than existing model compression methods and off-the-shelf compressed models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。