小模型经微调可高效生成正确语法的约束求解代码
Learn2Zinc: Fine-tuning Small Language Models for Text-to-Model Translation in MiniZinc
- 用多模型错误自举构建纠错数据集，提升小模型代码生成能力
- 微调后代码执行准确率最高达98%，但求解正确率仅35%
- 适合研究自然语言转领域专用代码的学者和开发者
大型语言模型在主流编程语言生成上表现优异，但在稀有领域特定语言如MiniZinc（组合优化问题的约束建模语言）上表现不佳。本文探究是否可通过针对性微调，使0.6B至20B参数的小型语言模型从自然语言问题描述生成语法正确且语义有效的MiniZinc模型。关键发现是：语法错误主导失败，如Qwen3、LLaMa、Gemma和GPT-OSS等模型在未微调时执行准确率接近零。我们提出跨模型错误自举方法，通过多个LLM运行收集语法错误，构建纠错训练数据集。该数据集使小模型在所有规模下均显著提升直接生成与思维链方法的表现。结合自我反思与集成策略，最高实现98%执行准确率。然而，求解准确率仍仅为35%，表明语法可学，但约束推理仍是挑战。我们开源了微调管道、数据集与模型，以推动文本到模型翻译的研究。
原文摘要 · Abstract (English)
Large language models excel at code generation for mainstream programming languages but struggle with rare, domain-specific languages such as MiniZinc, a constraint modeling language for combinatorial problems. We investigate whether targeted fine-tuning can teach small language models (0.6B to 20B parameters) to generate syntactically correct and semantically valid MiniZinc models from natural language problem descriptions. Our key finding is that syntax errors dominate failures when working with this domain specific language: the out-of-the-box execution accuracy of small language models such as Qwen3, LLaMa, Gemma, and GPT-OSS is near-zero. We propose a cross-model error bootstrapping approach that collects syntax errors from multiple LLM runs and leverage those to curate an error correction training dataset. This dataset allows us fine-tune small language models that consistently improves both direct code generation and chain-of-thought approaches across all model sizes. With self-reflection and ensembling, our approach achieves up to 98\% execution accuracy. In parallel, solution accuracy still remains at 35\%, indicating that while syntax is learnable, constraint reasoning remains a challenge. We contribute our fine-tuning pipeline, datasets, and models to opens-source for further research on text-to-model translation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。