用强化学习自动优化提示词，让大模型表现更稳定。
TAPR: Enhancing LLM Performance with a Task-Aware Prompt Rewriter
- 基于GRPO强化学习训练提示重写模型，目标是提升下游任务性能。
- 在Natural Questions和GSM8K上准确率显著提升，尤其在算术推理中增益明显。
- 适合希望简化提示工程、提升模型效果的非专家用户使用。
大型语言模型（LLMs）通常需要精心设计的提示才能发挥最佳性能，这对非专家用户构成障碍。本文提出任务感知提示重写器（TAPR），通过强化学习与组相对策略优化（GRPO）训练，将原始提示改写为任务优化后的提示，以提升下游模型表现。奖励信号来自大模型作为裁判对重写后提示及其对应输出的评估。在问答、摘要和算术推理等多样化任务上的实验表明，TAPR在提示重写能力上持续优于基线模型。对Phi-4-mini-instruct进行微调后生成的提示语言更清晰、指令更明确，在Natural Questions和GSM8K等基准测试中准确率显著提高。代码已开源：https://github.com/OliverSavolainen/task-specific-prompt-rewriter。
原文摘要 · Abstract (English)
Large Language Models (LLMs) often require carefully crafted prompts to unlock their full potential, which can be a barrier for non-expert users. This work addresses the challenge by introducing a Task-Aware Prompt Rewriter (TAPR), a model that reformulates user prompts into task-optimized prompts with the explicit goal of improving downstream LLM performance. We train TAPR using reinforcement learning with Group Relative Policy Optimization (GRPO), where rewards are derived from LLM-as-judge evaluations of both the reformulated prompt and the corresponding task output. Experimental results on diverse tasks, such as question answering, summarization, and arithmetic reasoning, show that our method yields consistent gains over base models in prompt rewriting ability. Fine-tuning Phi-4-mini-instruct (as the base model for TAPR) produces prompts that contain clearer and more instructive language, leading to higher accuracy on established benchmarks such as Natural Questions and GSM8K. Our code is available at: https://github.com/OliverSavolainen/task-specific-prompt-rewriter
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。