小模型生成结构化输出时，约束会降低答案正确率，需分开报告各项指标。
The Constraint Tax: Measuring Validity-Correctness Tradeoffs in Structured Outputs for Small Language Models
- 通过测量不同约束下的表现，揭示小模型在结构化输出中的有效性-正确性权衡。
- 硬约束使格式正确率升至100%，但答案准确率从19.7%降至11.0%。
- 建议系统分报格式正确率、答案准确率等四项指标，避免误导。
生产级大模型系统越来越需要机器可读的输出，如JSON对象、类型化轨迹、正则约束字段和工具调用模式。本文聚焦于本地部署与低成本小语言模型（SLM）场景，其中小于30亿参数的模型因隐私、延迟和通用硬件优势而受青睐，但其满足输出模式的能力有限。传统工程假设是硬性输出约束可提升可靠性而不影响答案质量，我们证明这一假设对小模型不成立。本文提出“约束税”测量协议，在固定模型、任务分布和问题实例下，隔离出结构化约束带来的答案与执行准确性损失。在15,000次基于商品级GPU的生成中，针对Qwen2.5-0.5B、Qwen2.5-1.5B和SmolLM2-1.7B模型，仅答案约束使格式有效率从61.5%提升至100.0%，但答案准确率从19.7%降至11.0%，错误但有效的输出比例从49.5%飙升至88.9%。最强行业类比为确定性日历工具调用任务：Qwen2.5-1.5B在仅提示生成JSON时可达到91.5%可执行准确率，但在相同硬性工具调用模式下仅48.0%，而两种模式均为100.0%格式有效。错误源于语义而非结构。我们还发现30亿参数边界仍存在直接约束税，且延迟封装支持一种有益设计模式：先自由推理，后晚约束。实践结论明确：生产系统应分别报告格式有效性、答案准确率、可执行准确率及错误但有效的输出比例。
原文摘要 · Abstract (English)
Production LLM systems increasingly require machine-readable outputs: JSON objects, typed traces, regex-constrained fields, and tool-call schemas. This paper targets on-device and low-cost small language model (SLM) deployments, where sub-3B models are attractive for privacy, latency, and commodity hardware but have limited capacity to satisfy schemas while solving tasks. The usual engineering assumption is that hard output constraints improve reliability without changing the underlying answer. We show that this assumption is unsafe for small models. We introduce \emph{constraint tax}, a measurement protocol for isolating the answer and executable-accuracy loss caused by structured-output constraints at fixed model, fixed task distribution, and fixed problem instances. Across 15,000 commodity-GPU generations with Qwen2.5-0.5B, Qwen2.5-1.5B, and SmolLM2-1.7B, hard answer-only schema decoding raises schema validity from 61.5\% to 100.0\%, but lowers answer accuracy from 19.7\% to 11.0\% and increases wrong-valid-schema outputs from 49.5\% to 88.9\%. The strongest industry analogue is a deterministic calendar tool-call task: Qwen2.5-1.5B achieves 91.5\% executable accuracy with prompt-only JSON but only 48.0\% under the same hard tool-call schema, while both modes are 100.0\% schema-valid. The error is semantic, not structural. We also show that the 3B boundary still pays a direct-schema tax and that delayed packaging supports a constructive design pattern: reason free, constrain late. The practical conclusion is direct: production systems should report schema validity, answer accuracy, executable accuracy, and wrong-valid-schema rate separately.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。