修复大模型对逻辑否定的过度保守时，提示设计不当反而降低准确率。
When Corrective Hints Hurt: Prompt Design in Reasoner-Guided Repair of LLM Overcaution on Entailed Negations under OWL~2~DL
- 用推理器校验答案并修正错误，但提示中加入开放世界假设提示会适得其反。
- 不带提示的修复模式准确率达97.8%，显著优于带提示的版本（67.2%）。
- 研究揭示提示框架比纠错内容更关键，适合评估大模型推理可靠性的人参考。
我们在GPT-5上发现一个可复现的错误模式：在OWL~2~DL合规查询中，当推理器结论为"no"时，模型常回答"unknown"，尤其在功能性属性闭包或类不交性条件下。基于180个经推理器审计的查询及2个独立领域（保险与临床）中18个手工编写的保留测试集，我们对比四种交互模式（单次、三次通用纠错重试、三次含开放世界假设提示的推理结果修复、同修复但无提示），在相同查询预算下进行评估。直接忠实度为43.9%（95%置信区间[36.8,51.2]）；通用重试达81.7%（[75.4,86.6]）；含提示修复为67.2%（[60.1,73.7]）；仅修复无提示达97.8%（[94.4,99.1]）。所有成对比较均在麦纳马拉精确检验与波弗罗尼校正下显著（α=0.01；所有p<10⁻⁵）。该错误模式在保留测试集中同样导致4/4错误。结论表明：提示框架的影响可能超过纠正内容本身，推理引导封装应明确消融分析。
原文摘要 · Abstract (English)
We report a reproducible error pattern in GPT-5.4 on OWL~2~DL compliance queries: the model frequently answers ``unknown'' when the reasoner-entailed answer is ``no'' under \emph{FunctionalProperty} closure or class \emph{disjointness}. Using 180 reasoner-audited queries from a procedural expansion of the observed pattern plus 18 hand-authored held-out queries in two unrelated domains (insurance and clinical), we compare four interaction modes under matched query budget: single-shot, three rounds of generic ``you-are-wrong'' retry, three rounds of reasoner-verdict repair with an open-world-assumption (OWA) hint, and the same repair without the hint. Direct faithfulness is 43.9\,\% (Wilson 95\,\% CI $[36.8,51.2]$); generic retry reaches 81.7\,\% ($[75.4,86.6]$); the verdict-with-hint variant is \emph{worse} at 67.2\,\% ($[60.1,73.7]$); the verdict-only variant reaches 97.8\,\% ($[94.4,99.1]$). All pairwise comparisons remain significant under McNemar's exact test with Bonferroni correction ($α= 0.01$; all $p < 10^{-5}$). The same fingerprint accounts for 4/4 errors on the held-out queries. Our interpretation is bounded: prompt framing can matter more than corrective content, and reasoner-guided wrappers should be ablated explicitly.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。