指导模型用特定符号格式输出，可提升分类准确率5%-10%。
Lost in Space: Finding the Right Tokens for Structured Output
- 按语义惯例设计输出格式，如选择题用字母、数值预测用数字
- 引导模型输出带前导空格的令牌，性能提升5%-10%
- 小模型更受益于前导空格，可缓解子词表示缺陷
通用语言模型虽擅长生成自然语言，但面对标注或分类任务时需更精确的结构化输出。当前大模型支持基于语法的格式约束采样，却可能意外降低下游性能。我们测试了四种主流模型家族在四个NLP基准上使用五种不同输出格式的表现。结果表明：当输出格式符合人类习惯（如多选题用字母、数值预测用实数）时，所有模型表现最优。此外，引导模型输出带前导空格的令牌可使性能提升5%-10%，小模型收益更显著。这得益于前导空格缓解了子词表示中的结构缺陷。本文提出零样本分类中使用结构化输出的最佳实践。
原文摘要 · Abstract (English)
General-purpose language models are trained to produce varied natural language outputs, but for some tasks, like annotation or classification, we need more specific output formats. LLM systems increasingly support structured output, which enforces formats by sampling tokens according to a grammar -- but also unpredictably reduces downstream performance. Are there systematic differences between grammars that appear semantically (and often visually) similar to humans? To answer this, we test four popular model families with five varying output formats on four common NLP benchmarks. We find all models perform most accurately when guided to use formats respecting convention, such as letters for multiple choice and real numbers for numerical prediction. Performance also improves by 5%-10% when guiding models to return tokens incorporating leading whitespace, with smaller models benefiting the most. We find leading whitespace helps models avoid structural deficiencies in subword token representations. We finally present best practices for researchers using language models as zero-shot classifiers with structured output.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。