新方法让大模型无需复杂提示，但越强模型越需简单提示
You Don't Need Prompt Engineering Anymore: The Prompting Inversion
- 用规则约束法减少思维链中的语义模糊和常识错误
- gpt-4o 上新方法准确率97%优于标准思维链的93%
- gpt-5 反而更差，说明强模型需更简单的提示
提示工程，尤其是思维链（CoT）提示，显著提升大模型的推理能力。本文提出‘雕琢’（Sculpting）方法，一种受约束的规则式提示策略，旨在改进标准 CoT，减少语义歧义和常识性错误。我们在三个 OpenAI 模型版本（gpt-4o-mini、gpt-4o、gpt-5）上，使用 GSM8K 数学推理基准（1,317 道题）评估三种提示策略（零样本、标准 CoT、Sculpting）。结果发现‘提示反转’现象：在 gpt-4o 上，Sculpting 准确率达 97%，优于标准 CoT 的 93%；但在 gpt-5 上反而下降至 94.00%，低于 CoT 的 96.36%。我们分析认为，这是从‘护栏’到‘枷锁’的转变所致——对中等模型有效的约束，在高级模型中引发过度字面化。详细错误分析表明，最优提示策略必须随模型能力协同进化，提示应随模型更强而变得更简单。
原文摘要 · Abstract (English)
Prompt engineering, particularly Chain-of-Thought (CoT) prompting, significantly enhances LLM reasoning capabilities. We introduce "Sculpting," a constrained, rule-based prompting method designed to improve upon standard CoT by reducing errors from semantic ambiguity and flawed common sense. We evaluate three prompting strategies (Zero Shot, standard CoT, and Sculpting) across three OpenAI model generations (gpt-4o-mini, gpt-4o, gpt-5) using the GSM8K mathematical reasoning benchmark (1,317 problems). Our findings reveal a "Prompting Inversion": Sculpting provides advantages on gpt-4o (97% vs. 93% for standard CoT), but becomes detrimental on gpt-5 (94.00% vs. 96.36% for CoT on full benchmark). We trace this to a "Guardrail-to-Handcuff" transition where constraints preventing common-sense errors in mid-tier models induce hyper-literalism in advanced models. Our detailed error analysis demonstrates that optimal prompting strategies must co-evolve with model capabilities, suggesting simpler prompts for more capable models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。