利用软提示的预映射结构,用更少查询实现更好指令优化。
PRESTO: Preimage-Informed Instruction Optimization for Prompting Black-Box LLMs
- 基于软提示的预映射共享评分,减少冗余查询。
- 在相同查询预算下,有效获得14倍的评分数据。
- 适合需要高效优化黑盒大模型指令的研究者。
大型语言模型(LLMs)在多个领域表现卓越,主要得益于其强大的指令遵循能力。针对无法访问内部参数的黑盒LLM,现有方法使用白盒LLM从优化后的软提示生成候选指令,但常出现不同软提示映射到同一指令的情况,导致查询冗余。本文将这种多对一映射重新解读为可利用的先验知识,提出PRESTO框架,通过三个核心组件提升效率:(1) 评分共享,对同一预映射内的所有软提示共享评估分数;(2) 基于预映射的初始化,利用预映射信息选择最大化搜索空间覆盖的初始点;(3) 评分一致性正则化,强制每个预映射内预测一致。实验表明,在33个指令优化任务上,PRESTO在相同查询预算下实现14倍的有效评分数据获取,显著提升优化效率。代码已公开于https://github.com/mlvlab/PRESTO。
原文摘要 · Abstract (English)
Large language models (LLMs) have achieved remarkable success across diverse domains, due to their strong instruction-following capabilities. This has led to increasing interest in optimizing instructions for black-box LLMs, whose internal parameters are inaccessible but widely used due to their strong performance. To optimize instructions for black-box LLMs, recent methods employ white-box LLMs to generate candidate instructions from optimized soft prompts. However, white-box LLMs often map different soft prompts to the same instruction, leading to redundant queries. While previous studies regarded this many-to-one mapping as a structure that hinders optimization efficiency, we reinterpret it as a useful prior knowledge that can accelerate the optimization. To this end, we introduce PREimage-informed inSTruction Optimization (PRESTO), a novel framework that leverages the preimage structure of soft prompts for efficient optimization. PRESTO consists of three key components: (1) score sharing, which shares the evaluation score with all soft prompts in a preimage; (2) preimage-based initialization, which selects initial data points that maximize search space coverage using preimage information; and (3) score consistency regularization, which enforces prediction consistency within each preimage. By leveraging preimages, PRESTO achieves the effect of effectively obtaining 14 times more scored data under the same query budget, resulting in more efficient optimization. Experimental results on 33 instruction optimization tasks demonstrate the superior performance of PRESTO. Code is available at https://github.com/mlvlab/PRESTO
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。