通过轻量适配器修正预训练模型的输出,提升部署后性能稳定性。
Adjusting Pretrained Backbones for Performativity
- 用浅层适配器对预训练主干进行贝叶斯最优标签偏移校正。
- 在视觉与语言任务中,使重训练过程损失更小,提前发现性能下降。
- 无需修改输入维度,适合已有模型快速适配新环境。
随着深度学习模型的广泛应用,它们会以多种方式影响部署环境,导致分布漂移,引发性能意外下降。现有预测性能的方法通常在预测向量中融入部署模型信息,虽具理论优势,但修改输入维度往往不实用。为此,我们提出一种模块化调整预训练主干以应对性能变化的新方法,提升样本效率并可复用现有深度学习资产。针对性能标签漂移问题,核心思想是训练一个浅层适配模块,基于待部署模型的充分统计量,对主干网络的原始输出(logits)执行贝叶斯最优标签偏移校正。该框架将输入特定特征嵌入与性能机制解耦。以动态基准测试为应用场景,我们在对抗性采样下评估了该方法在视觉和语言任务中的表现。结果表明,该方法能减小重训练过程中的损失,并有效筛选候选模型以预测性能退化。本工作为应对深度学习中的性能变化提供了首个基准方案。
原文摘要 · Abstract (English)
With the widespread deployment of deep learning models, they influence their environment in various ways. The induced distribution shifts can lead to unexpected performance degradation in deployed models. Existing methods to anticipate performativity typically incorporate information about the deployed model into the feature vector when predicting future outcomes. While enjoying appealing theoretical properties, modifying the input dimension of the prediction task is often not practical. To address this, we propose a novel technique to adjust pretrained backbones for performativity in a modular way, achieving better sample efficiency and enabling the reuse of existing deep learning assets. Focusing on performative label shift, the key idea is to train a shallow adapter module to perform a Bayes-optimal label shift correction to the backbone's logits given a sufficient statistic of the model to be deployed. As such, our framework decouples the construction of input-specific feature embeddings from the mechanism governing performativity. Motivated by dynamic benchmarking as a use-case, we evaluate our approach under adversarial sampling, for vision and language tasks. We show how it leads to smaller loss along the retraining trajectory and enables us to effectively select among candidate models to anticipate performance degradations. More broadly, our work provides a first baseline for addressing performativity in deep learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。