用小模型辅助大模型，提升不确定性判断能力
Asymmetric Duos: Sidekicks Improve Uncertainty
- 大模型配小模型，通过加权平均融合预测结果
- 仅增加10%-20%计算量，准确率与不确定性评估显著提升
- 适合资源有限但需可靠置信度的部署场景
在需要基于不确定性做出决策的场景中，传统方法依赖多次随机初始化训练的模型集成，但难以适应当前大规模模型与实际微调流程。本文提出一种新策略：将一个大型模型（如微调后的ViT-B）与一个更小但不那么准确的“侧伴”模型（如微调后的ResNet-34）配对，计算成本仅为前者的几分之一。通过简单的可学习加权平均聚合两者预测结果，令人惊讶的是，尽管存在不对称性，侧伴模型几乎不会损害主模型性能。在五个图像分类基准上，涵盖多种架构与训练方案（包括Soups），该方法在仅增加约10%-20%计算量的情况下，显著提升了准确率、不确定性量化及选择性分类表现。
原文摘要 · Abstract (English)
The go-to strategy to apply deep networks in settings where uncertainty informs decisions--ensembling multiple training runs with random initializations--is ill-suited for the extremely large-scale models and practical fine-tuning workflows of today. We introduce a new cost-effective strategy for improving the uncertainty quantification and downstream decisions of a large model (e.g. a fine-tuned ViT-B): coupling it with a less accurate but much smaller "sidekick" (e.g. a fine-tuned ResNet-34) with a fraction of the computational cost. We propose aggregating the predictions of this Asymmetric Duo by simple learned weighted averaging. Surprisingly, despite their inherent asymmetry, the sidekick model almost never harms the performance of the larger model. In fact, across five image classification benchmarks and a variety of model architectures and training schemes (including soups), Asymmetric Duos significantly improve accuracy, uncertainty quantification, and selective classification metrics with only ${\sim}10-20\%$ more computation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。