arXiv:2510.15992cs.LGcs.AI2025-10AAAI被引 1

Stratos自动化定制LLM蒸馏,高效适配云环境部署需求。

Stratos: An End-to-End Distillation Pipeline for Customized LLMs under Distributed Cloud Environments

  • 全自动选择服务器与师生模型对,动态调整蒸馏策略。
  • 在麻将推理任务上,学生模型准确率是GPT-4o的4倍。
  • 兼顾低延迟、低成本,适合垂直领域快速部署。

随着垂直领域任务和性能约束(如延迟与预算)的需求增长,定制化、低成本大语言模型的工业需求日益上升。知识蒸馏作为一种高效的模型压缩与迁移技术,提供了可行方案。然而,现有蒸馏框架常需人工干预,难以满足复杂的用户定义要求。为此,我们提出Stratos——一个面向分布式云环境的端到端大语言模型蒸馏管道,可自动完成服务器与模型选择、知识蒸馏及部署。在用户定义的性能与预算约束下,Stratos自动选取帕累托最优服务器,动态匹配师生模型对,并根据任务复杂度自适应蒸馏策略,以优化云部署效果。实验表明,针对罕见领域任务“麻将推理”,结合反向合成数据与知识注入,所生成的学生模型在准确率上达到其GPT-4o教师模型的4倍;同时实现更低延迟与成本,且不牺牲准确性。结果凸显其在垂直领域大模型部署中的潜力。

原文摘要 · Abstract (English)

The growing industrial demand for customized and cost-efficient large language models (LLMs) is fueled by the rise of vertical, domain-specific tasks and the need to optimize performance under constraints such as latency and budget. Knowledge distillation, as an efficient model compression and transfer technique, offers a feasible solution. However, existing distillation frameworks often require manual intervention and struggle to meet such complex user-defined distillation requirements. To bridge this gap, we propose Stratos, an end-to-end LLM distillation pipeline that automates server and model selection, knowledge distillation, and deployment in distributed cloud environments. Given user-defined constraints on model performance and system budget, Stratos automatically selects Pareto-optimal servers, dynamically matches teacher-student pairs, and adapts distillation strategies based on task complexity to optimize cloud hosting. Experiments show that Stratos produces a student model that achieves four times the accuracy of its GPT-4o teacher baseline on a rare, domain-specific Mahjong reasoning task with reverse synthetic data and knowledge injection. Moreover, it achieves reduced latency and cost without compromising accuracy. These results highlight its promise for vertical-domain LLM deployment.

大模型蒸馏云部署垂直领域自动化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。