用大模型的抽象概念提升小模型表现，无需重训练
Can abstract concepts from LLM improve SLM performance?
- 从大模型提取抽象概念向量，迁移到小模型推理中
- 在多种小模型上实现7%-15%准确率提升，尤其对Qwen3-0.6B有效
- 适合资源受限场景下优化小模型性能的研究者与工程师
大型语言模型（LLMs）在多样化任务中表现优异，但其在资源受限设备上的部署仍具挑战。现有压缩方法如量化、剪枝和蒸馏虽能减小内存占用，但常需大量实验和精细架构设计。本文利用已有技术从大模型中提取高层抽象概念（以控制向量表示），研究其在小语言模型（SLM）推理阶段的可迁移性。通过大量实验证明，这些概念可有效迁移至不同家族的小模型（如Phi、Llama、Qwen），并在多种任务中带来性能提升。此外，我们提出推理时动态调整控制强度的缩放策略，使Qwen3-0.6B模型的准确率提升了7%-15%。
原文摘要 · Abstract (English)
Large language models (LLMs) excel at diverse tasks, but their deployment on resource-constrained devices remains challenging. Existing methods like quantization, pruning, and distillation can reduce memory footprint but often demand extensive experimentation and careful infrastructure design. Leveraging existing techniques for extracting high-level concepts (represented as steering vectors) from larger models, we investigate their transferability to smaller language models (SLM) during inference. We demonstrate through extensive experimentation that these concepts can be effectively transferred to smaller models, irrespective of their family (e.g., Phi, Llama, Qwen), leading to performance improvements across a wide range of tasks. Furthermore, we introduce inference-time scaling to enhance performance by dynamically adjusting the steering intensity which has resulted in a 7-15\% of accuracy improvement for Qwen3-0.6B.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。