让大模型自动优化提示词,提升推理能力
RASPRef: Retrieval-Augmented Self-Supervised Prompt Refinement for Large Reasoning Models
- 用检索增强的自监督方式迭代优化提示词
- 在GSM8K上比静态提示词提升显著性能
- 无需人工标注,适合多任务通用优化
近期以DeepSeek R1和OpenAI o1为代表的推理型语言模型在GSM8K、MATH及多跳问答等结构化推理基准上表现优异。然而其性能对提示词设计高度敏感,而有效提示词的设计通常依赖人工且难以跨任务扩展。为此,我们提出检索增强的自监督提示词精炼框架RASPRef,无需人类标注或任务特定监督即可改进提示词。该方法通过检索相关示例与先前生成的推理轨迹,利用多样本一致性、验证器反馈及模型自动生成的批评信号,迭代优化提示词。与以往聚焦于改进模型输出的方法不同,RASPRef将提示词本身作为优化目标,通过检索引导的迭代过程实现提升。在GSM8K风格数学推理任务上的实验表明,检索引导提示显著优于静态提示基线。我们进一步探讨了检索质量、轨迹选择及自监督反馈信号对提示精炼效果的影响。结果表明,提示设计仍是推理型语言模型的关键因素,而自改善提示为提升推理性能提供了可扩展且实用的策略。
原文摘要 · Abstract (English)
Recent reasoning-focused language models such as DeepSeek R1 and OpenAI o1 have demonstrated strong performance on structured reasoning benchmarks including GSM8K, MATH, and multi-hop question answering tasks. However, their performance remains highly sensitive to prompt formulation, and designing effective prompts is typically a manual and iterative process that does not scale well across tasks or domains. To address this limitation, we introduce Retrieval-Augmented Self-Supervised Prompt Refinement (RASPRef), a framework that improves prompts without requiring human annotations or task-specific supervision. The approach retrieves relevant examples and previously generated reasoning trajectories, and leverages signals such as multi-sample consistency, verifier feedback, and model-generated critiques to iteratively refine the prompt. Unlike prior approaches that focus primarily on improving model outputs, RASPRef directly treats the prompt as the optimization target and improves it through an iterative retrieval-guided refinement process. Experiments on GSM8K-style mathematical reasoning tasks show that retrieval-guided prompting improves performance compared with a static prompting baseline. We further discuss how retrieval quality, trajectory selection, and self-supervised feedback signals may influence the effectiveness of prompt refinement. These findings suggest that prompt design remains a critical factor for reasoning-oriented language models, and that self-improving prompts offer a practical and scalable strategy for improving reasoning performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。