研究复杂指令叠加对大模型执行能力的影响，发现指令冲突会导致准确率暴跌至20%。
Instruction Stacking Collapse: A Benchmark and the Capability-Dependent Value of Prompt Compilation
- 通过逐层叠加24条验证过的指令，构建真实场景下的多约束评估基准。
- 指令叠加导致遵循率从96%降至最低20%，主要由成对冲突引发。
- 无需训练的指令编译器可提升弱模型性能，适合生产部署中的低算力场景。
实际应用中的提示词很少只包含单一指令，可能同时要求输出为合法JSON、控制字数、引用三篇文献并保持固定语气。本文研究此类约束累积对指令遵循能力的影响。提出一个基准，逐步叠加24条经验证器校验的指令（每次1到20条），评估三个生产级大模型（Claude Sonnet 4.6、GPT-5-mini、Gemini 2.5 Flash）。结果显示，指令遵循率呈非线性下降：从约96%降至最低20%，根源在于可复现的成对冲突——例如，仅“输出JSON”这一要求就与另外九项要求无法共存。随后评估一种免训练的修复方法：通过单次大模型调用重写堆叠提示，并在多个查询中复用。该方法收益具有能力依赖性：对较弱模型最高恢复11个百分点的遵循率，而强模型因已内化结构，基本不受影响。通过聚类稳健性测试、同基线对照和家族内缩放阶梯分析，确认增益源于重写机制本身，而非额外token、顺序调整或测量空间。论文发布完整基准、验证器及缓存运行结果以支持完全复现。
原文摘要 · Abstract (English)
Production prompts rarely carry a single instruction. One system message may require valid JSON, a word limit, three citations, and a fixed tone at the same time. We study how instruction-following degrades as such constraints accumulate. We introduce a benchmark that stacks 24 verifier-checked instructions, one to twenty at a time, and evaluate three production-tier LLMs (Claude Sonnet 4.6, GPT-5-mini, Gemini 2.5 Flash). Instruction-following degrades non-linearly: the follow rate falls from ~96% to as low as 20%, driven by a structured and reproducible set of pairwise conflicts. A single "output JSON" constraint, for example, is jointly unsatisfiable with nine others. We then evaluate a training-free remedy: an instruction compiler that rewrites the stacked prompt in a single LLM call and is reused across queries. Its benefit is capability-graded. It recovers up to +11 points of follow rate for weaker models, which are also the models most often deployed at scale, while leaving stronger models, which already internalise the same structure, essentially unchanged. Cluster-robust tests, same-baseline controls, and a within-family scaling ladder attribute the gain to the rewrite itself rather than to additional tokens, reordering, or measurement headroom. We release the benchmark, verifiers, and cached runs for full reproduction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。