无需验证集，用单样本预留法自动调优CLIP适配器混合比例。
Hold-One-Shot-Out (HOSO) for Validation-Free Few-Shot CLIP Adapters
- 用单样本预留集学习混合比例，训练与调参分离。
- 在11个数据集上平均性能超基线4个百分点以上。
- 适合追求严格少样本设置的视觉模型适配研究者。
许多CLIP适配方法依赖一个混合比例超参数，在通用预训练知识与少量样本的特定数据监督之间权衡。现有少样本CLIP适配技术通常通过测试集消融或额外验证集来选择该比例，无法严格满足少样本要求。本文提出一种简单、无需验证集的混合比例学习方法——持留一样本外（HOSO）。HOSO-Adapter在训练时使用一个样本作为预留集学习混合比例，其余样本用于适配器训练。在无验证的少样本协议下，HOSO-Adapter在11个标准少样本数据集上平均性能超越基线超过4个百分点。有趣的是，在8和16样本设置中，即使使用测试集最优混合比例，HOSO-Adapter仍表现更优。消融实验验证了单样本预留机制、解耦训练及对盲目学习基线的改进。代码已开源：https://github.com/chris-vorster/HOSO-Adapter
原文摘要 · Abstract (English)
In many CLIP adaptation methods, a blending ratio hyperparameter controls the trade-off between general pretrained CLIP knowledge and the limited, dataset-specific supervision from the few-shot cases. Most few-shot CLIP adaptation techniques report results by ablation of the blending ratio on the test set or require additional validation sets to select the blending ratio per dataset, and thus are not strictly few-shot. We present a simple, validation-free method for learning the blending ratio in CLIP adaptation. Hold-One-Shot-Out (HOSO) presents a novel approach for CLIP-Adapter-style methods to compete in the newly established validation-free setting. CLIP-Adapter with HOSO (HOSO-Adapter) learns the blending ratio using a one-shot, hold-out set, while the adapter trains on the remaining few-shot support examples. Under the validation-free few-shot protocol, HOSO-Adapter outperforms the CLIP-Adapter baseline by more than 4 percentage points on average across 11 standard few-shot datasets. Interestingly, in the 8- and 16-shot settings, HOSO-Adapter outperforms CLIP-Adapter even with the optimal blending ratio selected on the test set. Ablation studies validate the use of a one-shot hold-out mechanism, decoupled training, and improvements over the naively learnt blending ratio baseline. Code is released here: https://github.com/chris-vorster/HOSO-Adapter
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。