用小模型微调后，能像大模型一样教编程错误，还更便宜高效。
Narrowing the Gap: Supervised Fine-Tuning of Open-Source LLMs as a Viable Alternative to Proprietary Models for Pedagogical Tools
- 用4万条学生编译错误数据微调开源小模型，提升教学能力。
- 微调后小模型表现接近大模型，8000次测试中准确率达92%。
- 适合教育科技开发者，可复现，降低部署成本。
前沿大语言模型如ChatGPT和Gemini虽能解读初学者的编译错误，但因计算成本高、易过度协助，难以普及于教学场景。本文证明，通过监督微调（SFT）的小型专用模型是更可行的替代方案。我们构建了一个包含4万条真实初学编程（CS1/2）学生生成的C语言编译错误解释的新数据集，并用于微调Qwen3-4B、Llama-3.1-8B和Qwen3-32B三个开源模型。采用专家人工评估与基于验证过的LLM-as-judge集成的自动化大规模分析（8000条响应）相结合的双重评估方法。结果表明，微调显著提升了小模型的教学质量，其表现可媲美更大模型。我们分析了模型规模与质量之间的权衡，证实以高质量、领域特定数据微调紧凑高效的模型，是打造教育专用模型的有效策略。本文提供可复现的方法论，推动生成式AI在教育领域的广泛接入。
原文摘要 · Abstract (English)
Frontier Large language models (LLMs) like ChatGPT and Gemini can decipher cryptic compiler errors for novice programmers, but their computational scale, cost, and tendency to over-assist make them problematic for widespread pedagogical adoption. This work demonstrates that smaller, specialised language models, enhanced via Supervised Fine-Tuning (SFT), present a more viable alternative for educational tools. We utilise a new dataset of 40,000 C compiler error explanations, derived from real introductory programming (CS1/2) student-generated programming errors, which we used to fine-tune three open-source models: Qwen3-4B, Llama-3.1-8B, and Qwen3-32B. We performed a dual evaluation, combining expert human reviews with a large-scale automated analysis of 8,000 responses using a validated LLM-as-judge ensemble. Our results show that SFT significantly boosts the pedagogical quality of smaller models, achieving performance comparable to much larger models. We analyse the trade-offs between model size and quality, confirming that fine-tuning compact, efficient models on high-quality, domain-specific data is a potent strategy for creating specialised models to drive educational tools. We provide a replicable methodology to foster broader access to generative AI capabilities in educational contexts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。