用大模型生成的蓝图提升小模型推理能力，不增加模型大小。
Enhancing Reasoning Capabilities of Small Language Models with Blueprints and Prompt Template Search
- 用大模型生成结构化推理蓝图指导小模型
- 在GSM8K、MBPP等任务上显著提升小模型表现
- 无需训练，适合手机等资源受限设备
小语言模型（SLMs）为大型语言模型（LLMs）提供了高效替代方案。然而，其容量有限导致推理能力受限，且对提示变化敏感。为此，我们提出一种新框架，通过LLM生成的蓝图增强SLM的推理能力。这些蓝图提供结构化、高层次的推理指引，帮助SLMs系统性解决相关问题。此外，框架集成提示模板搜索机制，缓解SLMs对提示变化的敏感性。实验表明，该方法在多个任务上均提升SLM性能，包括数学（GSM8K）、编码（MBPP）和逻辑推理（BBH）。本方法无需增加模型尺寸或额外训练，为设备端或资源受限环境提供轻量、易部署的解决方案。
原文摘要 · Abstract (English)
Small language models (SLMs) offer promising and efficient alternatives to large language models (LLMs). However, SLMs' limited capacity restricts their reasoning capabilities and makes them sensitive to prompt variations. To address these challenges, we propose a novel framework that enhances SLM reasoning capabilities through LLM generated blueprints. The blueprints provide structured, high-level reasoning guides that help SLMs systematically tackle related problems. Furthermore, our framework integrates a prompt template search mechanism to mitigate the SLMs' sensitivity to prompt variations. Our framework demonstrates improved SLM performance across various tasks, including math (GSM8K), coding (MBPP), and logic reasoning (BBH). Our approach improves the reasoning capabilities of SLMs without increasing model size or requiring additional training, offering a lightweight and deployment-friendly solution for on-device or resource-constrained environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。