小模型可帮大模型高效选最优提示，省时省力
S2LPP: Small-to-Large Prompt Prediction across LLMs
- 用小模型预测大模型的提示效果，避免重复试错
- 在14个模型、多个任务上验证，性能接近人工最优
- 适合需要批量优化提示的NLP应用开发者
预训练大语言模型（LLMs）的性能对提示模板细节敏感，需大量人工调优，成本高昂。本研究在多个不同规模的LLM上进行实验，发现不同规模模型在提示偏好上具有一致性。该一致性不仅体现在问答任务，也适用于自然语言推理等其他任务。基于此，我们提出一种方法：用小模型为大模型筛选有效提示模板。实验表明，该方法显著降低提示工程成本，同时性能稳定接近最优提示。更重要的是，在14个不同LLM和广泛NLP任务上均验证有效，展现出强鲁棒性。
原文摘要 · Abstract (English)
The performance of pre-trained Large Language Models (LLMs) is often sensitive to nuances in prompt templates, requiring careful prompt engineering, adding costs in terms of computing and human effort. In this study, we present experiments encompassing multiple LLMs variants of varying sizes aimed at probing their preference with different prompts. Through experiments on Question Answering, we show prompt preference consistency across LLMs of different sizes. We also show that this consistency extends to other tasks, such as Natural Language Inference. Utilizing this consistency, we propose a method to use a smaller model to select effective prompt templates for a larger model. We show that our method substantially reduces the cost of prompt engineering while consistently matching performance with optimal prompts among candidates. More importantly, our experiment shows the efficacy of our strategy across fourteen LLMs and its applicability to a broad range of NLP tasks, highlighting its robustness
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。