用知识蒸馏优化提示词，让大模型更懂任务。
Automatic Prompt Optimization with Prompt Distillation
- 通过多阶段融合任务信息，用蒸馏压缩提示空间。
- 在文本分类与生成任务中平均提升20.12%性能。
- 适合想不用梯度就提升提示效果的研究者。
自动提示工程（Autoprompting）是为语言模型自动寻找最优提示词的技术，随着大语言模型研究的快速发展而日益受到关注。本文提出 DistillPrompt —— 一种基于大语言模型的新型自动提示方法，利用训练数据将任务特定信息分阶段融入提示中。该方法结合了知识蒸馏、压缩与聚合操作，更全面地探索提示空间。在 t-lite-instruct-0.1 模型上，于多个文本分类与生成任务的数据集上进行测试，结果表明其在关键指标上相比现有方法平均提升 20.12%（如与 Grips 相比），确立了 DistillPrompt 作为当前最有效的非梯度类自动提示方法之一。
原文摘要 · Abstract (English)
Autoprompting is the process of automatically selecting optimized prompts for language models, which is gaining popularity due to the rapid development of prompt engineering driven by extensive research in the field of large language models (LLMs). This paper presents DistillPrompt -- a novel autoprompting method based on large language models that employs a multi-stage integration of task-specific information into prompts using training data. DistillPrompt utilizes distillation, compression, and aggregation operations to explore the prompt space more thoroughly. The method was tested on different datasets for text classification and generation tasks using the t-lite-instruct-0.1 language model. The results demonstrate a significant average improvement (e.g., 20.12% across the entire dataset compared to Grips) in key metrics over existing methods in the field, establishing DistillPrompt as one of the most effective non-gradient approaches in autoprompting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。