用双模型蒸馏让边缘设备智能判断何时把任务丢给云端,省资源还更准。
Dual-Model Distillation for Efficient Action Classification with Hybrid Edge-Cloud Solution
- 通过双模型蒸馏生成数据,训练轻量级切换模型
- 在动作分类任务中降低计算开销并提升准确率
- 适合资源受限的边缘计算场景,如智能安防
随着大型视频-语言模型(VLMs)规模增大,其在实际应用中的部署面临硬件限制和计算成本高的挑战。为此,我们设计了一种混合边缘-云解决方案,利用小型模型进行本地处理,必要时将推理任务卸载至更大、更精确的云端模型。具体而言,提出一种新型无监督数据生成方法——双模型蒸馏(DMD),用于训练轻量级切换模型,该模型可预测边缘模型输出的不确定性,并在需要时选择性地将任务卸载到云端大模型。在动作分类任务上的实验结果表明,该框架不仅计算开销更低,且相比单独使用大模型的方案准确率更高。该框架为资源受限环境下的动作分类提供了可扩展、可适应的解决方案,具有超越医疗领域的潜在应用价值。值得注意的是,尽管本文中DMD生成的数据用于优化性能与资源使用,但其核心思想有望推动未来多模型间知识对齐的研究。
原文摘要 · Abstract (English)
As Artificial Intelligence models, such as Large Video-Language models (VLMs), grow in size, their deployment in real-world applications becomes increasingly challenging due to hardware limitations and computational costs. To address this, we design a hybrid edge-cloud solution that leverages the efficiency of smaller models for local processing while deferring to larger, more accurate cloud-based models when necessary. Specifically, we propose a novel unsupervised data generation method, Dual-Model Distillation (DMD), to train a lightweight switcher model that can predict when the edge model's output is uncertain and selectively offload inference to the large model in the cloud. Experimental results on the action classification task show that our framework not only requires less computational overhead, but also improves accuracy compared to using a large model alone. Our framework provides a scalable and adaptable solution for action classification in resource-constrained environments, with potential applications beyond healthcare. Noteworthy, while DMD-generated data is used for optimizing performance and resource usage in our pipeline, we expect the concept of DMD to further support future research on knowledge alignment across multiple models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。