让大模型先自由思考再格式化，能大幅减少输出错误。
The Format Tax
- 先自由生成再统一格式，避免格式要求干扰推理
- 格式请求本身就会导致准确率下降，与解码无关
- 适用于数学、科学、逻辑等多任务，适合开发应用者
要求大语言模型以JSON等结构化格式输出，本应是格式选择而非能力损耗。但我们发现，对JSON、XML、LaTeX、Markdown等格式的要求会显著降低开放权重模型的推理与写作性能。现有研究多聚焦于受限解码，但采样偏差仅解释部分损失。主要代价来自提示词本身：仅添加格式请求指令就造成大部分准确率下降，远在应用解码约束前发生。这一诊断揭示一个简单原则：将推理与格式分离。可通过先生成自由文本再重格式化，或在单次生成中支持扩展思考，从而显著恢复准确率。在六种开放权重模型、四种API模型、四种格式及涵盖数学、科学、逻辑和写作的多项任务中，该方法均有效恢复多数性能损失。值得注意的是，大多数近期闭源模型几乎不受格式税影响，说明问题不在于结构化生成本身，而是当前开源模型尚未弥补这一差距。代码已公开于 https://github.com/ivnle/the-format-tax。
原文摘要 · Abstract (English)
Asking a large language model to respond in JSON should be a formatting choice, not a capability tax. Yet we find that structured output requirements -- JSON, XML, LaTeX, Markdown -- substantially degrade reasoning and writing performance across open-weight models. The research response has focused on constrained decoding, but sampling bias accounts for only a fraction of the degradation. The dominant cost enters at the prompt: format-requesting instructions alone cause most of the accuracy loss, before any decoder constraint is applied. This diagnosis points to a simple principle: decouple reasoning from formatting. Whether by generating freeform first and reformatting in a second pass, or by enabling extended thinking within a single generation, separating the two concerns substantially recovers lost accuracy. Across six open-weight models, four API models, four formats, and tasks spanning math, science, logic, and writing, decoupling recovers most lost accuracy. Notably, most recent closed-weight models show little to no format tax, suggesting the problem is not inherent to structured generation but a gap that current open-weight models have yet to close. Code is available at https://github.com/ivnle/the-format-tax.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。