用大模型思维链训练小模型，提升工业设备健康监测精度。
Fine-Tuned Thoughts: Leveraging Chain-of-Thought Reasoning for Industrial Asset Health Monitoring
- 将大模型的思维链推理能力蒸馏到小模型中
- 微调后的小模型性能显著超越基础版本，接近大模型水平
- 适合资源受限但需复杂推理的工业场景应用
小型语言模型（SLMs）因高效、计算开销低且可微调，正日益应用于工业等专业领域，实现精准且低成本的解决方案。然而，在工业4.0等专业领域中，利用SLMs进行复杂推理仍具挑战。本文提出一种面向工业资产健康监测的知识蒸馏框架，通过链式思维（Chain-of-Thought, CoT）从大型语言模型（LLMs）向更高效的较小模型（SLMs）迁移推理能力。我们采用多选题问答（MCQA）提示方式蒸馏LLMs，并结合上下文学习验证生成知识质量，对比微调后的SLMs与主流LLMs的性能表现。实验结果表明，具备CoT推理能力的微调SLMs显著优于基线模型，大幅缩小与对应大模型间的差距。代码已开源：https://github.com/IBM/FailureSensorIQ。
原文摘要 · Abstract (English)
Small Language Models (SLMs) are becoming increasingly popular in specialized fields, such as industrial applications, due to their efficiency, lower computational requirements, and ability to be fine-tuned for domain-specific tasks, enabling accurate and cost-effective solutions. However, performing complex reasoning using SLMs in specialized fields such as Industry 4.0 remains challenging. In this paper, we propose a knowledge distillation framework for industrial asset health, which transfers reasoning capabilities via Chain-of-Thought (CoT) distillation from Large Language Models (LLMs) to smaller, more efficient models (SLMs). We discuss the advantages and the process of distilling LLMs using multi-choice question answering (MCQA) prompts to enhance reasoning and refine decision-making. We also perform in-context learning to verify the quality of the generated knowledge and benchmark the performance of fine-tuned SLMs with generated knowledge against widely used LLMs. The results show that the fine-tuned SLMs with CoT reasoning outperform the base models by a significant margin, narrowing the gap to their LLM counterparts. Our code is open-sourced at: https://github.com/IBM/FailureSensorIQ.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。