让小模型自己选答案，用大模型做筛选，效率更高。
Select to Think: Unlocking SLM Potential with Local Sufficiency
- 用大模型从候选词中挑最优解，简化监督信号。
- 1.5B小模型8个候选中95%包含32B大模型的正确答案。
- 无需调用大模型推理，小模型即可自主重排答案。
小语言模型（SLMs）部署高效，但推理能力常落后于大语言模型（LLMs）。现有方法要么在推理分歧点调用大模型，带来高延迟与成本；要么依赖标准蒸馏，受限于小模型对复杂生成分布的模仿能力。本文提出“局部充分性”：在分歧点，大模型首选词通常出现在小模型前K个预测中，即使未成为第一选择。据此提出Select to Think（S2T），将大模型角色从开放式生成转为从候选中选择，将监督信号简化为离散候选排序。进一步提出S2T-Local，将选择逻辑蒸馏至小模型，使其可自主重排而不依赖大模型推理。实验显示，1.5B SLM的前8个候选中，95%包含32B LLM的正确选择；S2T-Local使1.5B SLM在数学任务上相较贪心解码提升24.1%相对性能，仅需单轨迹即可达到8路径自洽的效能。
原文摘要 · Abstract (English)
Small language models (SLMs) offer efficient deployment, yet they often lag behind their larger counterparts (LLMs) in reasoning. Existing remedies either invoke an LLM at points of reasoning divergence, incurring substantial latency and cost, or rely on standard distillation, which is limited by the SLM's capacity to accurately mimic the LLM's complex generative distribution. We address this dilemma by identifying local sufficiency: at divergence points, the LLM's preferred token often resides within the SLM's top-K next-token predictions, even when failing to emerge as the SLM top-1 choice. We therefore propose Select to Think (S2T), which reframes the LLM's role from open-ended generation to selection among the SLM's proposals, simplifying the supervision signal to discrete candidate rankings. Leveraging this, we introduce S2T-Local, which distills the selection logic into the SLM, empowering it to perform autonomous re-ranking without inference-time LLM dependency. Empirically, a 1.5B SLM's top-8 candidates contain the 32B LLM's choice with a 95% hit rate, and S2T-Local improves the 1.5B SLM's Math Avg. over greedy decoding by 24.1% relative gain, matching the efficacy of 8-path self-consistency with single-trajectory efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。