改变结构化生成中键名措辞，能显著提升模型准确率
Schema Key Wording as an Instruction Channel in Structured Generation under Constrained Decoding
- 将模式键名作为隐式指令通道，与提示词共同引导生成
- 同一任务下更换键名可使准确率提升，模型表现受键名影响
- 适合关注提示工程与结构化输出设计的研究者参考
约束解码广泛用于使大语言模型生成满足模式（如 JSON）的结构化输出。现有工作主要将模式视为结构约束，忽略了模式键令牌进入自回归上下文可能带来的引导作用。据我们所知，这是首个系统研究约束解码下模式键作为隐式指令通道的工作。我们将结构化生成建模为多通道指令问题，任务信号可置于提示、模式键或两者。进一步提出投影感知分析：仅当键的语义增益超过语法约束投影带来的失真时，类 CoT 键才有效，解释了模型依赖性键效应。在数学推理基准上的实验表明，仅改变模式键措辞即可显著影响准确率，而保持提示、模型、输出结构和解码设置不变。Qwen 模型更受益于模式级指令，而 LLaMA 模型更依赖提示级引导，二者非加性交互。研究显示，模式设计不仅是输出格式，更是结构化生成中的指令组成部分。
原文摘要 · Abstract (English)
Constrained decoding is widely used to make large language models produce structured outputs that satisfy schemas such as JSON. Existing work mainly treats schemas as structural constraints, overlooking that schema-key tokens also enter the autoregressive context and may guide generation. To the best of our knowledge, we present the first systematic study of schema keys as an implicit instruction channel under constrained decoding. We formulate structured generation as a multi-channel instruction problem, where task signals can be placed in prompts, schema keys, or both. We further provide a projection-aware analysis: a CoT-style key helps only when its semantic gain exceeds the distortion induced by grammar-constrained projection, offering a theoretical explanation for model-dependent key effects. Experiments on mathematical reasoning benchmarks show that changing only schema-key wording can substantially affect accuracy while keeping the prompt, model, output structure, and decoding setup fixed. Qwen models tend to benefit more from schema-level instructions, whereas LLaMA models rely more on prompt-level guidance, and the two channels interact non-additively. Our findings show that schema design is not merely output formatting, but part of instruction specification in structured generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。