让视觉模型提前退出更安全，精准控制误判风险
SAFE-KD: Risk-Controlled Early-Exit Distillation for Vision Backbones
- 用分层知识蒸馏+共形风险控制，统一包裹主流视觉主干
- 提前退出样本的误判率可控，且在数据扰动下仍稳定
- 适合对可靠性要求高的实际部署场景
早期退出网络通过让简单输入提前终止来降低推理成本，但实际应用的关键在于判断何时退出是安全的。本文提出SAFE-KD，一种适用于现代视觉主干的通用多出口封装方法，结合分层知识蒸馏与共形风险控制（CRC）。在中间层添加轻量级退出头，通过解耦知识蒸馏（DKD）将强教师模型的知识传递至所有出口，并保证深层到浅层输出的一致性。推理时，基于保留集校准各出口的停止阈值，确保在可交换性假设下，用户指定的‘选择性’误判风险（仅针对提前退出样本）得到保障。在多个数据集和架构上，SAFE-KD实现了更优的精度-计算权衡、更强的校准性能，并在数据扰动下保持鲁棒，同时提供有限样本下的风险保证。
原文摘要 · Abstract (English)
Early-exit networks reduce inference cost by allowing ``easy'' inputs to stop early, but practical deployment hinges on knowing \emph{when} early exit is safe. We introduce SAFE-KD, a universal multi-exit wrapper for modern vision backbones that couples hierarchical distillation with \emph{conformal risk control}. SAFE-KD attaches lightweight exit heads at intermediate depths, distills a strong teacher into all exits via Decoupled Knowledge Distillation (DKD), and enforces deep-to-shallow consistency between exits. At inference, we calibrate per-exit stopping thresholds on a held-out set using conformal risk control (CRC) to guarantee a user-specified \emph{selective} misclassification risk (among the samples that exit early) under exchangeability. Across multiple datasets and architectures, SAFE-KD yields improved accuracy compute trade-offs, stronger calibration, and robust performance under corruption while providing finite-sample risk guarantees.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。