提出可解释的自动提示优化框架，解决黑箱问题并显著提升准确率
Reflection in the Dark: Exposing and Escaping the Black Box in Reflective Prompt Optimization
- 分拆生成与重写环节，用语义标签实现可追踪优化过程
- 在缺陷种子下准确率从13.50%恢复至87.57%，全面超越基线
- 适合需要可解释性与鲁棒性的大模型提示优化场景
自动提示优化（APO）已成为无需人工设计即可提升大模型性能的重要方法。以GEPA为代表的反思式APO通过诊断失败案例迭代优化提示，但其过程仍为黑箱且无标签，导致结果不可解释且易陷入系统性错误。我们实证揭示四类缺陷：在GSM8K数据集上使用有缺陷种子时，GEPA准确率从23.81%下降至13.50%。为此，我们提出VISTA——一种多智能体APO框架，将假设生成与提示重写解耦，实现语义标注的假设、并行小批量验证及可解释的优化轨迹。通过两层探索-利用机制（随机重启+epsilon-greedy采样），有效逃离局部最优。VISTA在相同缺陷种子下将准确率恢复至87.57%，并在所有条件下持续优于基线，覆盖GSM8K与AIME2025数据集。
原文摘要 · Abstract (English)
Automatic prompt optimization (APO) has emerged as a powerful paradigm for improving LLM performance without manual prompt engineering. Reflective APO methods such as GEPA iteratively refine prompts by diagnosing failure cases, but the optimization process remains black-box and label-free, leading to uninterpretable trajectories and systematic failure. We identify and empirically demonstrate four limitations: on GSM8K with a defective seed, GEPA degrades accuracy from 23.81% to 13.50%. We propose VISTA, a multi-agent APO framework that decouples hypothesis generation from prompt rewriting, enabling semantically labeled hypotheses, parallel minibatch verification, and interpretable optimization trace. A two-layer explore-exploit mechanism combining random restart and epsilon-greedy sampling further escapes local optima. VISTA recovers accuracy to 87.57% on the same defective seed and consistently outperforms baselines across all conditions on GSM8K and AIME2025.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。