解决提示词膨胀问题,让模型更准更快更短
ESPO: Error-Structured Prompt Optimization via Diagnose, Diversify, and Stabilize
- 分诊断、生成、选择三阶段优化提示词结构
- 在7个NLP任务上平均准确率提升3.76个百分点
- 适合追求高效精准提示工程的研究者和开发者
进化式提示优化器如GEPA存在提示词膨胀问题:每轮迭代添加规则与限制,导致提示词长度达原来的3倍,但准确率未提升。我们发现根源在于三点缺陷——错误观察不完整、搜索多样性不足、选择不可靠。为此提出ESPO(Error-Structured Prompt Optimization),将优化过程分为三个阶段:诊断阶段在一轮内将训练误差聚类为结构性模式;生成阶段通过四种互补策略产生候选提示,具备独立偏差;选择阶段采用自举稳定性选择。在七个公开NLP基准测试(Tweet、MMLU、GSM8K、HotpotQA、ScoNe、HoVer、PUPA)上,ESPO平均准确率较最先进方法(70.91%)提升3.76个百分点至74.67%,且在所有数据集上表现持平或超越,提示词长度缩短47%(1,004 vs 1,878字符),推理速度更快。跨模型实验在四个额外学生模型(Gemma 3 12B、Mistral 14B、Qwen3 32B、Claude Haiku 4.5)上均取得最高平均准确率,其中在Qwen3 GSM8K上提升达15.00%→91.40%。附录中的泛化界证明每个阶段对应测试时差距的一项成分,消融实验证明:仅增加多样性而不加自举选择反而降低性能(-1.20%)。
原文摘要 · Abstract (English)
Evolutionary prompt optimizers such as GEPA suffer from prompt bloat: each iteration appends rules and caveats, producing prompts up to 3$\times$ longer yet no more accurate. We trace this to three deficiencies - incomplete error observation, limited search diversity, and unreliable selection - and propose ESPO (Error-Structured Prompt Optimization), which decomposes prompt optimization into three phases: Diagnose clusters all training errors into structural patterns in one round; Propose generates candidates via four complementary strategies with independent biases; Select applies bootstrap stability selection. On seven public NLP benchmarks - Tweet, MMLU, GSM8K, HotpotQA, ScoNe, HoVer, and PUPA - ESPO improves average accuracy by $+$3.76 pp over the state-of-the-art (74.67% vs 70.91% for GEPA), matching or exceeding GEPA on every dataset while producing prompts 47% shorter (1,004 vs 1,878 chars) and faster at inference. Cross-model experiments across four additional student models (Gemma 3 12B, Mistral 14B, Qwen3 32B, Claude Haiku 4.5) show ESPO yields the best average accuracy on every model tested, with the largest gap on Qwen3 GSM8K (15.00% $\to$ 91.40%). A generalization bound (Appendix) grounds each phase in a corresponding term of the test-time gap, and the ablation confirms a key prediction: adding diversity without bootstrap selection actually hurts performance ($-$1.20%).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。