arXiv:2410.08601cs.CL2024-10EMNLP被引 21

用成功与失败案例指导优化提示,防止效果波动。

StraGo: Harnessing Strategic Guidance for Prompt Optimization

  • 结合正反案例提炼可操作优化策略
  • 在多个任务上达到提示优化新纪录
  • 适合需要稳定提示效果的工业应用

提示工程在各类应用中对大语言模型能力的发挥至关重要。现有提示优化方法虽能提升效果,但常引发提示漂移问题——新生成的提示可能损害原有有效案例的表现。同时,这些方法过度依赖大模型自身能力完成优化任务。本文提出StraGo(战略引导优化)方法,通过分析成功与失败案例,识别实现优化目标的关键因素,避免提示漂移。StraGo采用‘如何做’的方法论,利用上下文学习生成具体、可执行的优化策略,提供分步指导。在推理、自然语言理解、领域知识及工业应用等多类任务上的大量实验表明,StraGo性能优越,建立了提示优化的新基准,展现出稳定且高效的提示改进能力。

原文摘要 · Abstract (English)

Prompt engineering is pivotal for harnessing the capabilities of large language models (LLMs) across diverse applications. While existing prompt optimization methods improve prompt effectiveness, they often lead to prompt drifting, where newly generated prompts can adversely impact previously successful cases while addressing failures. Furthermore, these methods tend to rely heavily on LLMs' intrinsic capabilities for prompt optimization tasks. In this paper, we introduce StraGo (Strategic-Guided Optimization), a novel approach designed to mitigate prompt drifting by leveraging insights from both successful and failed cases to identify critical factors for achieving optimization objectives. StraGo employs a how-to-do methodology, integrating in-context learning to formulate specific, actionable strategies that provide detailed, step-by-step guidance for prompt optimization. Extensive experiments conducted across a range of tasks, including reasoning, natural language understanding, domain-specific knowledge, and industrial applications, demonstrate StraGo's superior performance. It establishes a new state-of-the-art in prompt optimization, showcasing its ability to deliver stable and effective prompt improvements.

提示工程大模型优化策略

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。