研究发现，模型对提示和模式描述的依赖程度因模型而异，模式设计比指令位置更重要。
Your Prompt Is Not the Only Prompt: How Much Do LLMs Weight Structured-Output Schema Descriptions?
- 对比提示与模式描述的指令效果，发现模式设计影响模型表现。
- 冲突时模式指令可导致准确率下降45个百分点，甚至超过提示指令的影响。
- 增加中间推理字段能提升15-24分，证明结构设计是关键杠杆。
结构化输出中，大模型根据预定义的JSON模式填充数据，但模式描述本身也构成第二条指令通道。本研究在单字段分类任务上，通过十种不同模型配置测试了标签定义应置于系统提示、用户提示还是模式描述中的效果。结果表明，模式描述并未始终优于提示放置；对于GPT-4.1和GPT-5.4（无推理），模式放置比系统提示低11-13个百分点。然而，模式并非静态元数据：当提示与模式冲突时，错误的模式指令导致准确率下降5-45个百分点，例如Claude Haiku 4.5从52.5%降至7%，GPT-5.5从100%降至73%。此外，在标签前添加强制中间推理字段后，模式仅使用场景下准确率提升15-24个百分点，且在所有测试中均超越仅用系统提示的表现。该效应在Claude Sonnet 4.6中于中等推理水平仍显著，说明单纯扩展思考无法达到相同收益。这表明模式设计会影响模型对字段描述信息的利用效率。整体而言，模式影响具有模型依赖性。实践中，系统提示仍是安全默认选择，但更关键的是统一指令源、避免提示/模式漂移。更重要的是，模式设计本身可能是比指令位置更强的控制杠杆。从业者应将提示与模式视为统一指令界面，并针对目标模型实证验证指令位置与字段设计。
原文摘要 · Abstract (English)
Structured output, where an LLM populates a predefined JSON schema, has become a default mechanism for data labeling and information extraction, but it also introduces a second instruction channel through schema descriptions. We tested whether classification-label definitions are better placed in the system prompt, user prompt, or schema description using a single-field classification task with nonce labels across ten model configurations from two vendors. Schema descriptions did not consistently outperform prompt-based placement; for GPT-4.1 and GPT-5.4 without reasoning, schema placement underperformed system prompts by 11-13 percentage points. Yet schemas are not inert metadata: when prompts and schemas conflicted, incorrect schema instructions caused accuracy drops of 5-45 points, with Claude Haiku 4.5 falling from 52.5% to 7%, indicating that schema instructions can override prompt instructions, and GPT-5.5 falling from 100% to 73%. Further, adding a required intermediate reasoning field before the label field improved schema-only accuracy by 15-24 points when headroom existed, exceeding system-prompt-only performance in every case tested. The effect held even for Claude Sonnet 4.6 at medium reasoning, where extended thinking alone did not produce a comparable gain. This suggests that schema design can affect how effectively models use information encoded in field descriptions. Overall, these results indicate that schema influence is model-dependent. In practice, the system prompt remains a safe default for definitions, but the bigger discipline is maintaining a single source of truth and preventing prompt/schema drift. More importantly, schema design itself may be a stronger lever than instruction placement. Practitioners should treat prompts and schemas as a unified instruction surface and empirically validate both placement and field design for their target model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。