让大模型选更差的答案,反而能更准评估回答质量
GRP: Goal-Reversed Prompting for Zero-Shot Evaluation with LLMs
- 反向提问:让模型选出更差的回答,再通过排除法判断优劣
- 在GPT-4o数据上,准确率从61.71%提升至66.23%,最高增6个百分点
- 对强模型效果更明显,适合提升推理与数学类任务评估精度
成对的LLM作为裁判评估要求模型判断两个候选回答中哪个更好。本文提出一种仅需修改一句话的简单方法:让裁判选出更差的回答,再通过排除法还原偏好,称为目标反转提示(GRP)。该方法无需额外推理轮次,兼容任意提示模板(直接、思维链或Arena-Hard SOP),且不影响原有评估流程。两个观察驱动该设计:人类解题常采用逆向思维,而当前指令微调的裁判存在正向偏见,反向提问可抵消此偏差。在JudgeBench严格一致性协议下(仅当两种顺序判断均符合真实偏好才算正确),所有测试的闭源裁判均有提升。使用GPT-4o生成的回答对时,Arena-Hard SOP基线在GPT-4o上的准确率从61.71%升至66.23%(+4.52),Claude-3.5-Sonnet从60.00%升至66.00%(+6.00),尤其在推理与数学任务上提升最大。即使使用Claude-3.5-Sonnet生成的数据或简化为直接提示模板,性能提升依然存在,表明目标反转作用于底层判断行为而非特定评分标准。更强模型获益更多,说明该方法揭示了额外推理能力而非弥补缺陷。
原文摘要 · Abstract (English)
Pairwise LLM-as-a-judge evaluation asks the judge to identify the \emph{better} of two candidate answers. We study a one-line modification that asks for the \emph{worse} answer instead and recovers the preference by elimination, a procedure we call Goal-Reversed Prompting (GRP). GRP introduces no extra inference rounds, composes with any prompt template (direct, chain-of-thought, or Arena-Hard SOP), and leaves the rest of the evaluation pipeline untouched. Two observations motivate the reversal. Reverse reasoning is a recurring strategy in human problem solving, and modern instruction-tuned judges exhibit a positive-leaning bias that asking for the worse answer can counteract. On JudgeBench under a strict consistency protocol that counts a judgment as correct only when both response orderings agree with the gold preference, GRP improves all three closed-source judges we test across both response-pair sources. With GPT-4o-generated pairs, the Arena-Hard SOP baseline improves from 61.71\% to 66.23\% for GPT-4o (+4.52) and from 60.00\% to 66.00\% for Claude-3.5-Sonnet (+6.00), with the largest absolute gains on Reasoning and Mathematics. The lift persists when response pairs come from Claude-3.5-Sonnet and when the SOP scaffolding is stripped to a minimal direct-prompting template, suggesting that goal reversal acts on the underlying judging behavior rather than on a particular rubric. Stronger judges benefit more than weaker ones, suggesting that goal reversal exposes additional reasoning capacity rather than compensating for its absence.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。