让模型更聪明地提前退出，提升推理速度同时保持准确率。
COSEE: Consistency-Oriented Signal-Based Early Exiting via Calibrated Sample Weighting Mechanism
- 基于样本权重校准，让每个分类器关注可能提前退出的样本。
- 在GLUE上实现更高性能与效率的平衡，支持灵活加速比。
- 适合追求高效推理的NLP应用，尤其适配多场景部署。
早退出是通过动态调整每个样本执行的层数来提升预训练语言模型（PLMs）推理效率的有效方法。然而，现有大多数方法在训练时对易样本和难样本一视同仁，忽略了测试时的早退出行为，导致训练与测试不一致。尽管部分方法在固定加速比下解决了该问题，但如何在灵活调整加速比的同时保持训练与测试的一致性仍缺乏研究。为此，我们提出一种新的一致性导向信号早退出框架（COSEE），通过校准样本权重机制，使每个分类器在不同加速场景下更关注那些更可能在此层退出的样本。在GLUE基准上的大量实验表明，COSEE在多种退出信号和模型架构下均有效，实现了性能与效率的更好权衡。
原文摘要 · Abstract (English)
Early exiting is an effective paradigm for improving the inference efficiency of pre-trained language models (PLMs) by dynamically adjusting the number of executed layers for each sample. However, in most existing works, easy and hard samples are treated equally by each classifier during training, which neglects the test-time early exiting behavior, leading to inconsistency between training and testing. Although some methods have tackled this issue under a fixed speed-up ratio, the challenge of flexibly adjusting the speed-up ratio while maintaining consistency between training and testing is still under-explored. To bridge the gap, we propose a novel Consistency-Oriented Signal-based Early Exiting (COSEE) framework, which leverages a calibrated sample weighting mechanism to enable each classifier to emphasize the samples that are more likely to exit at that classifier under various acceleration scenarios. Extensive experiments on the GLUE benchmark demonstrate the effectiveness of our COSEE across multiple exiting signals and backbones, yielding a better trade-off between performance and efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。