通过对比模型两次推理差异，自动优化提示词提升推理能力。
ContraPrompt: Contrastive Prompt Optimization via Dyadic Reasoning Trace Analysis
- 分析模型失败与成功时的完整推理过程差异，提取优化信号。
- 在多个基准测试中超越现有方法，最高提升20.8%相对效果。
- 无需人工标注，适合需要高精度推理的应用场景。
现有提示词优化方法要么孤立分析单次失败，要么比较不同提示在多例上的表现，均无法获取同一输入下成功与失败之间的推理过程差异。本文提出ContraPrompt，基于一个关键观察：当模型在反馈后重试成功时，其两次思维链轨迹的差异构成了一种未被先前方法捕捉的优化信号。不同于以往对比方法，我们对比完整的中间推理过程——两段轨迹共享模型、输入和基础提示，剩余差异反映的是推理策略与附加错误反馈的区别，称为二元推理轨迹分析。多轮求解阶段是一个可编程的智能体重试循环，能自动生成对比数据而无需人工标注。提取出的规则被组织成基于输入特征的决策树，用于路由指令。在四个推理与合规基准测试上，ContraPrompt在所有任务上均优于GEPA（Agrawal et al., 2026），绝对提升分别为：HotPotQA +8.29 pp（+20.8% rel.）、GDPR-Bench +2.21 pp（+18.2% rel.）、GPQA Diamond +7.14 pp（+10.6% rel.）、BBH +0.74 pp（+0.85% rel.）。消融实验表明，二元轨迹对比是核心组件，移除后平均相对性能下降16%。在53个EvalSet黑盒优化问题中，ContraPrompt在同等预算下击败GEPA 11次，持平41次，仅1次落后。在FiNER-139金融命名实体识别任务（Loukas et al., 2022）上，相较未优化基线提升+7.77 pp（+11.6% rel.），比GEPA提升+1.94 pp（+2.66% rel.），且分支条件与美国公认会计准则（US GAAP）金融工具分类一致。
原文摘要 · Abstract (English)
Prompt optimization methods either analyze individual failures in isolation or compare prompt variants across examples, operating on single execution traces with no access to the reasoning process distinguishing success from failure on the same input. We introduce ContraPrompt, built on the observation that when a model fails but succeeds on a retry with feedback, the difference between its two chain-of-thought traces constitutes an optimization signal not captured by prior methods. Unlike prior contrastive methods, we compare complete intermediate reasoning processes: the two traces share model, input, and base prompt, so remaining differences reflect reasoning strategy and appended error feedback -- we call this dyadic reasoning trace analysis. The multi-attempt solving phase is an instrumented agentic retry loop that generates contrastive data automatically without human annotation. Extracted rules are organized into an input-aware decision tree routing instructions by observable input characteristics. On four reasoning and compliance benchmarks, ContraPrompt outperforms GEPA (Agrawal et al., 2026) on all four, with absolute gains of +8.29 pp on HotPotQA (+20.8% rel.), +2.21 pp on GDPR-Bench (+18.2% rel.), +7.14 pp on GPQA Diamond (+10.6% rel.), and +0.74 pp on BBH (+0.85% rel.). Ablations confirm dyadic trace contrastivity is the critical component, with a -16% relative average drop upon its removal. On 53 EvalSet black-box optimization problems, ContraPrompt beats GEPA on 11, ties on 41, and loses on 1 at equal budget. On FiNER-139 financial named entity recognition (Loukas et al., 2022), ContraPrompt achieves +7.77 pp over the unoptimized baseline (+11.6% rel.) and +1.94 pp over GEPA (+2.66% rel.), with branch conditions aligning with standard US GAAP financial-instrument categories.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。