arXiv:2601.08412cs.AI2026-01

让小模型学会大模型的编程推理能力,实现无人机轻量化智能控制

Hybrid Distillation with CoT Guidance for Edge-Drone Control Code Generation

  • 用思维链引导+混合蒸馏,把大模型的推理过程迁移到小模型
  • 蒸馏后模型在多SDK任务上准确率超90%,推理速度提升6倍
  • 适合资源受限的无人机边缘计算场景,可直接部署

大型语言模型在代码生成中展现巨大潜力,但其高资源消耗与无人机平台对实时性、轻量化的严苛要求存在矛盾。本文提出融合知识蒸馏、思维链引导与监督微调的集成方法,实现复杂推理与代码生成能力向小型模型的有效迁移。首先构建覆盖主流无人机SDK的高质量数据集,包含指令-代码-思维链三元组,并引入反事实负样本增强训练;其次以量化后的DeepSeek-Coder-V2-Lite为教师模型,采用黑盒与白盒混合蒸馏策略生成高质量思维链软标签,结合加权交叉熵损失(硬标签)进行联合训练;最后通过面向无人机控制场景的提示调优工程,提升对SDK类型识别与函数调用匹配的能力。实验表明,蒸馏后的轻量模型在保持超过90%代码生成准确率的同时,部署效率和推理速度显著提升,验证了该方法在实现精确、轻量级无人机智能控制方面的可行性与优势。

原文摘要 · Abstract (English)

With large language models demonstrating significant potential in code generation tasks, their application to onboard control of resource-constrained Unmanned Aerial Vehicles has emerged as an important research direction. However, a notable contradiction exists between the high resource consumption of large models and the real-time, lightweight requirements of UAV platforms. This paper proposes an integrated approach that combines knowledge distillation, chain-of-thought guidance, and supervised fine-tuning for UAV multi-SDK control tasks, aiming to efficiently transfer complex reasoning and code generation capabilities to smaller models. Firstly, a high-quality dataset covering various mainstream UAV SDKs is constructed, featuring instruction-code-reasoning chains, and incorporates counterfactual negative samples for data augmentation, guiding the model to learn the end-to-end logic from instruction parsing to code generation. Secondly, leveraging DeepSeek-Coder-V2-Lite quantized via QLoRA as the teacher model, and based on a hybrid black-box and white-box distillation strategy, high-quality chain-of-thought soft labels are generated. These are combined with a weighted cross-entropy loss using hard labels to transfer complex reasoning capabilities to the smaller student model. Finally, through prompt tuning engineering optimized for the UAV control scenario, the model performance on core tasks such as SDK type recognition and function call matching is enhanced. Experimental results indicate that the distilled lightweight model maintains high code generation accuracy while achieving significant improvements in deployment and inference efficiency, effectively demonstrating the feasibility and superiority of our approach in achieving precise and lightweight intelligent control for UAVs

代码生成知识蒸馏无人机控制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。