让AI提示词学会总结经验,持续优化推理能力。
Reflection-Enhanced Meta-Optimization Integrating TextGrad-style Prompt Optimization with Memory-Driven Self-Evolution
- 用记忆笔记式检索增强生成,记录并复用过往优化经验
- 在GSM8K数学题上比TextGrad更稳定,泛化能力更强
- 适合需要长期改进提示词策略的智能系统研发
近期的提示词优化方法(如TextGrad)可通过类梯度方式自动调整文本提示,提升大语言模型在特定下游任务上的表现。然而,现有方法通常无状态,各次优化独立进行,缺乏历史经验的保存与利用机制,且易过拟合,导致提示更新泛化能力差。为此,我们提出反射增强型元优化框架REMO,融合(1)基于记忆的“错误笔记”式检索增强生成模块,和(2)由大模型驱动的自适应元控制器,可整合多轮优化中的反思性洞察,迭代优化整体提示策略。该架构不仅支持细粒度提示调优,还实现跨轮次知识积累与复用,支持持续改进。我们在Qwen3-32B标准推理模式下(无显式思维链提示)评估REMO在GSM8K数学推理基准上的表现。实验表明,相比TextGrad基线,REMO展现出更稳定、更鲁棒的泛化性能,尽管计算开销更高。我们详细阐述算法设计,开展定性和定量分析,并通过全面消融实验验证各组件贡献。
原文摘要 · Abstract (English)
Recent advances in prompt optimization, exemplified by methods such as TextGrad, enable automatic, gradient-like refinement of textual prompts to enhance the performance of large language models (LLMs) on specific downstream tasks. However, current approaches are typically stateless and operate independently across optimization runs, lacking mechanisms to preserve and leverage historical optimization experience. Furthermore, they are susceptible to overfitting, often yielding prompt updates that generalize poorly beyond the immediate task context. To address these limitations, we propose Reflection-Enhanced Meta-Optimization (REMO), a novel framework that integrates (1) a memory-augmented Reflection Retrieval-Augmented Generation (RAG) module - structured as a "mistake notebook" and (2) a Self-Adaptive Optimizer, implemented via an LLM-driven meta-controller that synthesizes epoch-level reflective insights to iteratively improve system-level prompting strategies. This architecture enables not only local, fine-grained prompt tuning akin to TextGrad, but also the systematic accumulation and reuse of cross-run optimization knowledge, thereby supporting continual improvement over time. We instantiate the REMO framework using Qwen3-32B in standard inference mode - without explicit chain-of-thought prompting - and evaluate its efficacy on the GSM8K benchmark for mathematical reasoning. Experimental results demonstrate that, compared to a TextGrad baseline, REMO achieves more stable and robust generalization, albeit at the cost of increased computational overhead. We provide a detailed exposition of the algorithmic design, conduct a qualitative and quantitative analysis of optimization dynamics, and present a comprehensive ablation study to elucidate the contributions of each component.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。