arXiv:2509.00509cs.CV2025-09

用API模型蒸馏出专用分割模型,仅靠输出标签就能高效训练。

Make me an Expert: Distilling from Generalist Black-Box Models into Specialized Models for Semantic Segmentation

  • 通过注意力图动态选最优分辨率进行伪标签生成
  • 在多个数据集上实现显著性能提升,仅需一热预测接口
  • 适合无权值访问、只能调用API的实用场景

人工智能即服务(AIaaS)通过API提供预训练模型,但现有领域自适应方法难以应对黑盒模型——无法获取权重、训练数据或对数输出。为此,我们提出黑盒蒸馏(B2D)设置,约束为:(1)API模型为开放词汇且基于大规模通用数据训练;(2)仅可访问一热预测结果。我们发现,开放词汇模型对输入分辨率敏感,不同物体类别在不同尺度下表现最佳,称为“分辨率困境”。为此提出ATGC方法,利用DINOv2注意力图计算熵值,识别信息量高的尺度用于伪标签生成,实现有效蒸馏。实验表明,在多个数据集上均取得显著提升,且仅需一热预测。代码已开源:https://github.com/yasserben/ATGC。

原文摘要 · Abstract (English)

The rise of Artificial Intelligence as a Service (AIaaS) democratizes access to pre-trained models via Application Programming Interfaces (APIs), but also raises a fundamental question: how can local models be effectively trained using black-box models that do not expose their weights, training data, or logits, a constraint in which current domain adaptation paradigms are impractical ? To address this challenge, we introduce the Black-Box Distillation (B2D) setting, which enables local model adaptation under realistic constraints: (1) the API model is open-vocabulary and trained on large-scale general-purpose data, and (2) access is limited to one-hot predictions only. We identify that open-vocabulary models exhibit significant sensitivity to input resolution, with different object classes being segmented optimally at different scales, a limitation termed the "curse of resolution". Our method, ATtention-Guided sCaler (ATGC), addresses this challenge by leveraging DINOv2 attention maps to dynamically select optimal scales for black-box model inference. ATGC scores the attention maps with entropy to identify informative scales for pseudo-labelling, enabling effective distillation. Experiments demonstrate substantial improvements under black-box supervision across multiple datasets while requiring only one-hot API predictions. Our code is available at https://github.com/yasserben/ATGC.

模型蒸馏黑盒学习语义分割

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。