用一次API调用初始化本地编码器,后续无需再调用接口即可高效适配大模型。
Prime Once, then Reprogram Locally: An Efficient Alternative to Black-Box Service Model Adaptation
- 先用一次API调用训练本地轻量编码器,再在本地进行白盒微调。
- 在GPT-4o上比零样本提升27.8%,且比传统方法减少99.99%的API调用。
- 适合需要频繁适配封闭模型但受限于调用成本的开发者使用。
适配闭源服务模型(如API)通常依赖零阶优化(ZOO),但该方法需大量昂贵的API调用,且优化缓慢不稳定。我们观察到现代API(如GPT-4o)对输入扰动不敏感,导致ZOO效果下降。为此,提出一种高效替代方案AReS:仅通过一次与API交互,训练一个可响应的本地预训练编码器,仅微调其轻量顶层。随后所有适配与推理均在本地完成,彻底消除后续API开销。实验表明,在十种数据集上,AReS优于现有方法(视觉语言模型+2.5%,标准模型+15.6%),在GPT-4o上较零样本提升27.8%,而API调用减少超99.99%。
原文摘要 · Abstract (English)
Adapting closed-box service models (i.e., APIs) for target tasks typically relies on reprogramming via Zeroth-Order Optimization (ZOO). However, this standard strategy is known for extensive, costly API calls and often suffers from slow, unstable optimization. Furthermore, we observe that this paradigm faces new challenges with modern APIs (e.g., GPT-4o). These models can be less sensitive to the input perturbations ZOO relies on, thereby hindering performance gains. To address these limitations, we propose an Alternative efficient Reprogramming approach for Service models (AReS). Instead of direct, continuous closed-box optimization, AReS initiates a single-pass interaction with the service API to prime an amenable local pre-trained encoder. This priming stage trains only a lightweight layer on top of the local encoder, making it highly receptive to the subsequent glass-box (white-box) reprogramming stage performed directly on the local model. Consequently, all subsequent adaptation and inference rely solely on this local proxy, eliminating all further API costs. Experiments demonstrate AReS's effectiveness where prior ZOO-based methods struggle: on GPT-4o, AReS achieves a +27.8% gain over the zero-shot baseline, a task where ZOO-based methods provide little to no improvement. Broadly, across ten diverse datasets, AReS outperforms state-of-the-art methods (+2.5% for VLMs, +15.6% for standard VMs) while reducing API calls by over 99.99%. AReS thus provides a robust and practical solution for adapting modern closed-box models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。