用大模型提炼小模型纠错经验,生成可复用的编程提醒提示词。
Distilling Reasoning Traces into Advisory Prompts for Software Engineering Tasks
- 通过大模型分析小模型思考模式下的纠错案例,提炼通用规则。
- 生成的简短提醒提示词能有效提升小模型在编程任务中的准确率。
- 适合资源受限的模型部署场景,尤其适用于编程教育与代码生成。
语言模型广泛用于代码生成与处理(如识别代码幻觉、推测输入或预测输出),但其仍可能出错,且错误后果严重。主要原因是训练数据多为人类编写的不完美代码,难以获得完全无缺陷的大规模代码语料。因此,无需额外训练的推理阶段纠错方法尤为重要。尽管‘思考模式’可降低错误,但会增加计算开销。本文受人类编程学习过程启发:识别错误、反思认知失误、归纳通用规则并内化为指导原则。我们首先在低资源语言模型中识别‘思考模式’成功避免错误的案例,再由更大模型分析这些案例,生成解释性总结,并进一步压缩为简洁的‘建议提示词’。该方法适用于多种中小型模型,且部分提示词可迁移至其他模型。我们还分析了语言模型常见的编码错误类型,并界定该方法适用的情境。
原文摘要 · Abstract (English)
Language models are widely used for generating and otherwise processing code (e.g., identifying code hallucinations, possible inputs, or predicting outputs); however, LLMs can make mistakes, which can be serious. One key issue is that models are trained on (still) largely human-written, and thus imperfect, code; it's not easy to find sufficiently large code corpora that are entirely free of bugs. Thus, other inference-time ways of reducing LLM errors, without additional training, are desirable. "Reasoning" or "thinking" modes, exposed as a togglable feature by hybrid reasoning models, do reduce errors; however, reasoning consumes additional resources. This paper asks if better performance can be achieved without always incurring the cost of reasoning. Human students of programming learn to avoid mistakes by (a) identifying them, (b) reflecting upon the cognitive lapses that led to them (essentially, "thinking through" the errors), (c) inferring general rules or lessons from these reflections, and (d) internalizing these lessons into rules. In tutorial sessions with an instructor, this is a common Socratic interaction. Examples of such internalizable rules might include the nugget "Before coding, restate the requirements to clarify them." Inspired by this process, this paper describes an approach where we first identify examples in which "thinking mode" in a (low-resource) LLM avoids errors. These errors, and their avoidance via "thinking" in the same LLM, are then examined by a bigger LLM to generate summary explanations; these are then summarized by a large LLM into brief advisory prompts. This approach works on many modest-sized models; in some cases, the "advisory prompts" thus learned can also be gainfully transferred to other models. We also present investigations into the nature of coding errors that language models make, and a characterization of when this approach can be helpful.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。