研究如何将大模型压缩为可部署在可穿戴设备上的小模型
Towards Characterizing Knowledge Distillation of PPG Heart Rate Estimation Models
- 用四种知识蒸馏方法压缩大模型，适配边缘设备
- 发现模型大小与性能存在可预测的缩放规律
- 适合做可穿戴生理信号实时检测的模型设计
通过智能手表等可穿戴设备获取的光电容积脉搏波描记（PPG）信号进行心率估计，对个人健康具有重要意义。尽管已有深度学习模型在该任务上表现优异，但要将其部署于可穿戴设备，仍需满足严格的内存和延迟限制。本文探索并分析了如何将大型预训练PPG模型压缩为适合边缘设备实时推理的小型模型。我们通过系统性地测试教师与学生模型容量组合，评估了四种蒸馏策略：硬标签蒸馏、软标签蒸馏、解耦知识蒸馏（DKD）和特征蒸馏。研究揭示了模型规模与性能之间的缩放规律，为构建可预测、实用的边缘可部署生理感知模型奠定了基础。
原文摘要 · Abstract (English)
Heart rate estimation from photoplethysmography (PPG) signals generated by wearable devices such as smartwatches and fitness trackers has significant implications for the health and well-being of individuals. Although prior work has demonstrated deep learning models with strong performance in the heart rate estimation task, in order to deploy these models on wearable devices, these models must also adhere to strict memory and latency constraints. In this work, we explore and characterize how large pre-trained PPG models may be distilled to smaller models appropriate for real-time inference on the edge. We evaluate four distillation strategies through comprehensive sweeps of teacher and student model capacities: (1) hard distillation, (2) soft distillation, (3) decoupled knowledge distillation (DKD), and (4) feature distillation. We present a characterization of the resulting scaling laws describing the relationship between model size and performance. This early investigation lays the groundwork for practical and predictable methods for building edge-deployable models for physiological sensing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。