用提示词提炼大模型推理逻辑，小模型零成本复现高精度推理。
Prompt-Level Distillation: A Non-Parametric Alternative to Model Fine-Tuning for Efficient Reasoning
- 从大模型提取推理模式，转化为小模型可读的结构化提示。
- 小模型在多个数据集上性能接近大模型，如StereoSet F1达90%。
- 无需微调，逻辑透明，适合金融、法律等高监管场景。
高级推理通常依赖思维链提示（Chain-of-Thought prompting），虽准确但带来高昂延迟和测试时推理开销。现有替代方案是微调小型模型，但常牺牲可解释性并引入显著资源与运维负担。为此，我们提出提示级蒸馏（Prompt-Level Distillation, PLD）。该方法从教师模型中提取显式推理模式，并组织为结构化表达指令，嵌入学生模型的系统提示中。在Gemma-3 4B上评估显示，PLD使StereoSet的宏平均F1从57%提升至90.0%，Contract-NLI从67%升至83%，LogiQA准确率达70%。在Mistral Small 3.1上的相似结果表明其跨架构泛化能力，使紧凑模型以极低延迟实现前沿性能。这些表达性指令使决策过程完全可人工验证，适用于法律、金融、内容审核等监管严苛领域，以及高吞吐和边缘设备部署。
原文摘要 · Abstract (English)
Advanced reasoning typically requires Chain-of-Thought prompting, which is accurate but incurs prohibitive latency and substantial test-time inference costs. The standard alternative, fine-tuning smaller models, often sacrifices interpretability while introducing significant resource and operational overhead. To address these limitations, we introduce Prompt-Level Distillation (PLD). We extract explicit reasoning patterns from a Teacher model and organize them into a structured list of expressive instructions for the Student model's System Prompt. Evaluated using Gemma-3 4B, PLD improved Macro F1 scores on StereoSet (57\% to 90.0\%) and Contract-NLI (67\% to 83\%), while increasing LogiQA accuracy to 70\%. Similar results on Mistral Small 3.1 demonstrate cross-architecture generalizability, enabling these compact models to match frontier performance with negligible latency overhead. These expressive instructions render the decision-making process transparent, allowing for full human verification of logic, making this approach ideal for regulated industries such as law, finance, and content moderation, as well as high-volume use cases and edge devices.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。