文本梯度是自动提示优化的错误类比，研究揭示其原理不成立。
Textual Gradients are a Flawed Metaphor for Automatic Prompt Optimization
- 用文本梯度类比优化提示，但实验显示该类比不准确。
- 虽能提升模型性能，但机制与梯度无关。
- 适合关注提示优化原理或设计新方法的研究者。
一个精心设计的提示可以提升大语言模型的表现；自动提示优化技术旨在无需人工调参即可提高性能。一类主流方法引入了‘文本梯度’的类比。我们通过一系列实验和案例研究考察了这些文本梯度方法的行为。尽管这些方法通常能带来性能提升，但实验表明，梯度类比并不能准确解释其实际行为。这些发现有助于指导提示优化策略的选择，并推动新方法的发展。
原文摘要 · Abstract (English)
A well-engineered prompt can increase the performance of large language models; automatic prompt optimization techniques aim to increase performance without requiring human effort to tune the prompts. One leading class of prompt optimization techniques introduces the analogy of textual gradients. We investigate the behavior of these textual gradient methods through a series of experiments and case studies. While such methods often result in a performance improvement, our experiments suggest that the gradient analogy does not accurately explain their behavior. Our insights may inform the selection of prompt optimization strategies, and development of new approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。