小模型推理时调用结构化步骤指导，提升复杂问题解决能力
Big Reasoning with Small Models: Instruction Retrieval at Inference Time
- 通过聚类训练题生成可复用的推理指南，替代原始文本检索
- 在医学、法律、数学领域分别提升9.4%、7.9%、5.1%准确率
- 无需微调，适合资源受限场景下的高精度推理任务
小型语言模型（SLMs）具备低成本、私密性好、支持本地部署的优势，但在需要专业领域知识或多步推理的问题上表现不佳。现有方法要么依赖模型规模（如思维链提示），要么需针对特定任务训练以限制通用性（如知识蒸馏），或检索非结构化信息，仍需小模型自行设计推理策略。本文提出推理指令检索，一种推理时干预机制，通过向小模型注入结构化的可复用推理流程而非原始文本片段来增强其能力。我们构建了指令语料库：对相似训练问题进行聚类，并使用教师模型生成结合领域背景与明确步骤的通用指引。推理时，小模型检索与当前查询最相关的指令并执行对应流程，无需额外微调。在医学、法律、数学三个挑战性领域中，3B及以上参数模型均获得稳定提升，准确率分别提高9.4%、7.9%、5.1%，其中最强的14B模型在知识密集型任务上超越GPT-4o的零样本性能。
原文摘要 · Abstract (English)
Small language models (SLMs) enable low-cost, private, on-device inference, but they often fail on problems that require specialized domain knowledge or multi-step reasoning. Existing approaches for improving reasoning either rely on scale (e.g., chain-of-thought prompting), require task-specific training that limits reuse and generality (e.g., distillation), or retrieve unstructured information that still leaves the SLM to determine an appropriate reasoning strategy. We propose instruction retrieval, an inference-time intervention that augments an SLM with structured, reusable reasoning procedures rather than raw passages. We construct an Instruction Corpus by clustering similar training questions and using a teacher model to generate generalizable guides that pair domain background with explicit step-by-step procedures. At inference, the SLM retrieves the instructions most relevant to a given query and executes the associated procedures without any additional fine-tuning. Across three challenging domains: medicine, law, and mathematics, instruction retrieval yields consistent gains for models with at least 3B parameters, improving accuracy by 9.4%, 7.9%, and 5.1%, respectively, with the strongest 14B model surpassing GPT-4o's zero-shot performance on knowledge-intensive tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。