复杂提示会削弱结构化推理，导致准确率从100%降至0%-30%
Prompt Complexity Dilutes Structured Reasoning: A Follow-Up Study on the Car Wash Problem
- 在复杂提示中加入多重指令，干扰了先推理后结论的顺序
- 仅用STAR框架时准确率达100%，在生产级提示中降至0%-30%
- 模型能正确推理但因提前输出结论而失败，适合调试提示设计者
先前研究[Jo, 2026]表明，STAR推理（情境、任务、行动、结果）将Claude Sonnet 4.5上洗车问题的准确率从0%提升至85%，添加额外提示层后达100%。本跟进研究探讨：在包含60多行、经多次迭代优化的InterviewMate生产级提示中，STAR是否仍有效？在Claude Sonnet 4.6上测试三种条件，每组20次：(A)含Anthropic个人资料的生产提示，(B)含默认个人资料的生产提示，(C)原始纯STAR提示。结果显示，C组得分为100%（经n=100验证），而A组和B组分别为0%和30%。提示复杂度稀释了结构化推理效果。原因在于如“以具体信息开头”等指令强制结论先行，逆转了使STAR有效的推理-结论顺序。一例中，模型先输出“简答：走。”，随后才正确识别约束条件——证明其具备正确推理能力，却已错误定论。跨模型对比显示，纯STAR提示在无修改下，从Sonnet 4.5的85%提升至Sonnet 4.6的100%，表明模型升级可增强孤立环境下的结构化推理。结果表明，不能假设结构化推理框架在复杂提示环境中依然有效。推理与结论的顺序是提示设计的关键变量。
原文摘要 · Abstract (English)
In a previous study [Jo, 2026], STAR reasoning (Situation, Task, Action, Result) raised car wash problem accuracy from 0% to 85% on Claude Sonnet 4.5, and to 100% with additional prompt layers. This follow-up asks: does STAR maintain its effectiveness in a production system prompt? We tested STAR inside InterviewMate's 60+ line production prompt, which had evolved through iterative additions of style guidelines, format instructions, and profile features. Three conditions, 20 trials each, on Claude Sonnet 4.6: (A) production prompt with Anthropic profile, (B) production prompt with default profile, (C) original STAR-only prompt. C scored 100% (verified at n=100). A and B scored 0% and 30%. Prompt complexity dilutes structured reasoning. STAR achieves 100% in isolation but degrades to 0-30% when surrounded by competing instructions. The mechanism: directives like "Lead with specifics" force conclusion-first output, reversing the reason-then-conclude order that makes STAR effective. In one case, the model output "Short answer: Walk." then executed STAR reasoning that correctly identified the constraint -- proving the model could reason correctly but had already committed to the wrong answer. Cross-model comparison shows STAR-only improved from 85% (Sonnet 4.5) to 100% (Sonnet 4.6) without prompt changes, suggesting model upgrades amplify structured reasoning in isolation. These results imply structured reasoning frameworks should not be assumed to transfer from isolated testing to complex prompt environments. The order in which a model reasons and concludes is a first-class design variable.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。