用小模型自动优化大模型提示，提升复杂任务表现
Prompt-R1: Collaborative Automatic Prompting Framework via End-to-end Reinforcement Learning
- 小模型通过强化学习生成优化提示，与大模型协作推理
- 在多个数据集上超越基线模型，显著提升准确率和推理质量
- 即插即用框架，支持多种大模型，适合需要高效提示工程的场景
近期，大型语言模型（LLMs）发展迅速。然而，在面对复杂问题时，大多数用户难以提供准确有效的提示，限制了LLM性能。为此，我们提出Prompt-R1，一种端到端强化学习框架，利用小规模LLM与大规模LLM协作，替代用户交互以更高效地解决问题。该协作被建模为多轮提示交互，其中小规模LLM负责思考与生成提示，大规模LLM执行复杂推理。设计双约束奖励机制，优化正确性、生成质量和推理准确性。Prompt-R1提供即插即用框架，支持多种大规模LLM的推理与训练。在多个公开数据集上的实验表明，Prompt-R1在各类任务中显著优于基线模型。代码已开源：https://github.com/QwenQKing/Prompt-R1。
原文摘要 · Abstract (English)
Recently, advanced large language models (LLMs) have emerged at an increasingly rapid pace. However, when faced with complex problems, most users are often unable to provide accurate and effective prompts to interact with LLMs, thus limiting the performance of LLMs. To address this challenge, we propose Prompt-R1, an end-to-end reinforcement learning framework that uses a small-scale LLM to collaborate with large-scale LLMs, replacing user interaction to solve problems better. This collaboration is cast as a multi-turn prompt interaction, where the small-scale LLM thinks and generates prompts, and the large-scale LLM performs complex reasoning. A dual-constrained reward is designed to optimize for correctness, generation quality, and reasoning accuracy. Prompt-R1 provides a plug-and-play framework that supports both inference and training with various large-scale LLMs. Experiments on multiple public datasets show that Prompt-R1 significantly outperforms baseline models across tasks. Our code is publicly available at https://github.com/QwenQKing/Prompt-R1.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。