arXiv:2509.02093cs.CLcs.AI2025-09被引 2

通过对比优质与劣质示例,让模型自动优化提示词

Better by Comparison: Retrieval-Augmented Contrastive Reasoning for Automatic Prompt Optimization

  • 从数据集中检索参考示例,用对比推理改进提示词
  • 在HelpSteer2上比基线提升显著,多维度评估表现更优
  • 适合想提升提示工程效果的研究者和开发者

自动提示优化近年成为提升大语言模型提示质量的策略,目标是生成更准确、更有用的响应。然而,以往工作多聚焦于直接优化提示或微调模型,忽略了利用大语言模型固有的推理能力,通过对比示例进行学习的潜力。本文提出对比推理提示优化(CRPO),将提示优化建模为检索增强的推理过程。该方法从HelpSteer2数据集(一个开源数据集,每条响应均标注了有用性、正确性、连贯性、复杂度和冗余度)中检索前k个参考提示-响应对,并构建两种互补优化范式:(1) 分层对比推理,让模型比较高质量、中等质量和低质量的示例(提示与响应),通过反思式推理优化自身生成;(2) 多指标对比推理,让模型分析各评价维度下的最优示例,并融合其优势生成优化提示。通过显式对比高低质量示例,CRPO使模型能推断出某些提示成功或失败的原因,从而实现更鲁棒、可解释的优化。在HelpSteer2基准上的实验表明,CRPO显著优于基线。研究结果凸显了对比、检索增强推理在推进自动提示优化中的潜力。

原文摘要 · Abstract (English)

Automatic prompt optimization has recently emerged as a strategy for improving the quality of prompts used in Large Language Models (LLMs), with the goal of generating more accurate and useful responses. However, most prior work focuses on direct prompt refinement or model fine-tuning, overlooking the potential of leveraging LLMs' inherent reasoning capability to learn from contrasting examples. In this paper, we present Contrastive Reasoning Prompt Optimization (CRPO), a novel framework that formulates prompt optimization as a retrieval-augmented reasoning process. Our approach retrieves top k reference prompt-response pairs from the HelpSteer2 dataset, an open source collection where each response is annotated for helpfulness, correctness, coherence, complexity, and verbosity, and constructs two complementary optimization paradigms: (1) tiered contrastive reasoning, where the LLM compares high-, medium-, and low-quality exemplars (both prompts and responses) to refine its own generation through reflective reasoning, and (2) multi-metric contrastive reasoning, where the LLM analyzes the best exemplars along each evaluation dimension and integrates their strengths into an optimized prompt. By explicitly contrasting high and low quality exemplars, CRPO enables the model to deduce why certain prompts succeed while others fail, thereby achieving more robust and interpretable optimization. Experimental results on the HelpSteer2 benchmark demonstrate that CRPO significantly outperforms baselines. Our findings highlight the promise of contrastive, retrieval-augmented reasoning for advancing automatic prompt optimization.

提示优化对比学习检索增强

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。