用伪代码训练大模型，让其更准确理解指令。
Training with Pseudo-Code for Instruction Following
- 在训练时将自然语言指令转为伪代码，增强模型对结构化指令的理解。
- 在12个基准上提升指令遵循能力，最高相对增益达21%。
- 适合需要精准执行复杂指令的AI应用开发者。
尽管大型语言模型（LLMs）能力快速提升，但在处理包含组合结构的简单明确指令时仍表现不佳。研究表明，用伪代码表达指令可提升模型遵循效果。然而，编写伪代码繁琐，依赖少样本示例或推理时提示代码对非专业用户不自然。为此，我们提出一种训练时方法：使用包含自然语言指令及其对应伪代码表示和最终响应的指令微调数据来微调LLMs。我们在涵盖指令遵循、数学推理和常识推理的12个公开基准上，针对六种基础模型进行评估。结果表明，采用伪代码训练的模型在指令遵循任务中表现更可靠，相对提升8-21%，同时在数学与常识推理任务中基本保持甚至部分提升性能，所有评估基准平均提升最高达30%。
原文摘要 · Abstract (English)
Despite rapid advances in the capabilities of Large Language Models (LLMs), they continue to struggle with following relatively simple and unambiguous instructions, particularly when compositional structure is involved. Recent work suggests that models may follow instructions more effectively when they are expressed in pseudo-code rather than natural language. However, writing pseudo-code programs can be tedious, and relying on few-shot demonstrations or inference-time code prompting is often unnatural for non-expert users of LLMs. To overcome these limitations, we propose a training time approach that fine-tunes LLMs using instruction-tuning data augmented with pseudo-code representations of natural language instructions paired with final responses. We evaluate our method on 12 publicly available benchmarks spanning instruction-following, mathematical reasoning, and commonsense reasoning, across six base models. Our results show that models trained with pseudo-code follow instructions more reliably, achieving relative gains of 8-21\% on instruction following benchmarks, while largely preserving and in some cases improving performance on mathematical and commonsense reasoning tasks, with an average gain of up to 30\% across all evaluated benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。