arXiv:2505.04016cs.CLcs.AI2025-05EMNLP被引 27

让大模型输出更规范,用轻量模型自动纠正格式偏差。

SLOT: Structuring the Output of Large Language Models

  • 用微调的小模型作后处理,适配各种大模型和输出结构。
  • 在标准测试中实现99.5%的结构准确率和94.0%的内容相似度。
  • 小模型加SLOT可媲美大厂闭源模型,适合资源受限场景。

结构化输出对大语言模型在智能体和信息提取等关键应用中至关重要。尽管具备强大能力,大模型常生成不符合预定义模式的输出,严重阻碍可靠应用开发。本文提出SLOT(Structured LLM Output Transformer),一种与模型无关的方法,将非结构化输出转换为精确的结构化格式。不同于依赖约束解码或绑定特定模型的现有方案,SLOT采用微调的轻量语言模型作为后处理层,实现对多种大模型和模式规格的灵活适配。我们构建了系统化的数据准备与合成流程,并设计了正式评估方法,量化衡量模式准确率与内容保真度。实验表明,微调后的Mistral-7B搭配约束解码可达到99.5%的模式准确率和94.0%的内容相似度,显著优于Claude-3.5-Sonnet(分别高出25和20个百分点)。值得注意的是,即使是小型模型Llama-3.2-1B,在使用SLOT后也能达到甚至超过更大规模的专有模型的结构化生成能力,支持在资源受限环境下实现可靠结构化输出。

原文摘要 · Abstract (English)

Structured outputs are essential for large language models (LLMs) in critical applications like agents and information extraction. Despite their capabilities, LLMs often generate outputs that deviate from predefined schemas, significantly hampering reliable application development. We present SLOT (Structured LLM Output Transformer), a model-agnostic approach that transforms unstructured LLM outputs into precise structured formats. While existing solutions predominantly rely on constrained decoding techniques or are tightly coupled with specific models, SLOT employs a fine-tuned lightweight language model as a post-processing layer, achieving flexibility across various LLMs and schema specifications. We introduce a systematic pipeline for data curation and synthesis alongside a formal evaluation methodology that quantifies both schema accuracy and content fidelity. Our results demonstrate that fine-tuned Mistral-7B model with constrained decoding achieves near perfect schema accuracy (99.5%) and content similarity (94.0%), outperforming Claude-3.5-Sonnet by substantial margins (+25 and +20 percentage points, respectively). Notably, even compact models like Llama-3.2-1B can match or exceed the structured output capabilities of much larger proprietary models when equipped with SLOT, enabling reliable structured generation in resource-constrained environments.

结构化输出后处理轻量模型格式纠错

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。