提出两阶段数据筛选框架,提升边缘设备训练效率与精度。
A Two-Stage Data Selection Framework for Data-Efficient Model Training on Edge Devices
- 分两阶段筛选重要数据:粗粒度过滤+细粒度最优选择
- 实测训练时间减少43%,最终准确率提升6.2%
- 适合资源受限的边缘计算场景,兼顾效率与效果
由于数据隐私和个性化服务需求,边缘设备上的机器学习模型训练需求日益增长。然而当前设备端训练面临数据利用率低的问题,主要源于训练吞吐量低、存储有限以及数据重要性差异大。为提高数据利用效率,我们提出两阶段数据筛选框架 { t Titan},从流式数据中选出对模型训练最具价值的数据批次,同时保证训练效率与效果。第一阶段通过粗粒度方式筛选出潜在高价值候选数据集;第二阶段采用理论最优策略,精准识别能带来最大模型性能提升的数据批次。为进一步提升时间和资源效率,{ t Titan} 采用流水线设计,实现数据筛选与模型训练并行执行,利用空闲计算资源避免资源冲突。我们在真实边缘设备上对三个代表性任务(涵盖不同模型与数据模态)进行了评估。结果表明,{ t Titan} 在系统开销微小(如数据处理延迟、内存占用、能耗)的前提下,可实现最高达43%的训练时间减少和6.2%的最终准确率提升。
原文摘要 · Abstract (English)
The demand for machine learning (ML) model training on edge devices is escalating due to data privacy and personalized service needs. However, we observe that current on-device model training is hampered by the under-utilization of on-device data, due to low training throughput, limited storage and diverse data importance. To improve data resource utilization, we propose a two-stage data selection framework {\sf Titan} to select the most important data batch from streaming data for model training with guaranteed efficiency and effectiveness. Specifically, in the first stage, {\sf Titan} filters out a candidate dataset with potentially high importance in a coarse-grained manner.In the second stage of fine-grained selection, we propose a theoretically optimal data selection strategy to identify the data batch with the highest model performance improvement to current training round. To further enhance time-and-resource efficiency, {\sf Titan} leverages a pipeline to co-execute data selection and model training, and avoids resource conflicts by exploiting idle computing resources. We evaluate {\sf Titan} on real-world edge devices and three representative edge computing tasks with diverse models and data modalities. Empirical results demonstrate that {\sf Titan} achieves up to $43\%$ reduction in training time and $6.2\%$ increase in final accuracy with minor system overhead, such as data processing delay, memory footprint and energy consumption.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。