用强化学习生成精简指令，让大模型快速适配新任务
Prompt-MII: Meta-Learning Instruction Induction for LLMs
- 通过强化学习训练模型自动生成简洁有效的任务指令
- 在90个未见任务上提升F1 4-9点，仅需原上下文1/3到1/13的token
- 适合需要快速部署、资源受限的模型应用场合
将大语言模型适配新任务的常用方法是上下文学习（ICL），但随着上下文长度增加，推理成本显著上升。本文提出一种指令归纳方法，从训练样本中提炼出紧凑且描述性强的提示，使模型性能接近使用完整训练集的ICL。具体提出PROMPT-MII框架，基于强化学习元学习一个可即时生成紧凑指令的模型，适用于任意新数据集。在超过3000个来自HuggingFace的多样化分类数据集上训练，并在90个未见任务上评估。结果表明，该方法在下游任务中提升F1 4-9点（相对提升10%-20%），性能媲美ICL，同时减少3-13倍的输入token数。
原文摘要 · Abstract (English)
A popular method to adapt large language models (LLMs) to new tasks is in-context learning (ICL), which is effective but incurs high inference costs as context length grows. In this paper we propose a method to perform instruction induction, where we take training examples and reduce them to a compact but descriptive prompt that can achieve performance comparable to ICL over the full training set. Specifically, we propose PROMPT-MII, a reinforcement learning (RL) based framework to meta-learn an instruction induction model that can generate compact instructions on the fly for an arbitrary new dataset. We train on over 3,000 diverse classification datasets from the HuggingFace hub, and evaluate on 90 unseen tasks. PROMPT-MII improves downstream model quality by 4-9 F1 points (10-20% relative), matching ICL performance while requiring 3-13x fewer tokens.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。