arXiv:2508.16647cs.LG2025-08被引 2

针对边缘设备训练数据采样偏差问题,提出自适应优化方法提升模型精度。

AdapSNE: Adaptive Fireworks-Optimized and Entropy-Guided Dataset Sampling for Edge DNN Training

  • 用烟花算法优化非单调搜索,抑制降维后异常点。
  • 引入熵引导机制实现均匀采样,避免样本代表性偏差。
  • 设计专用加速器降低边缘计算能耗与面积开销。

在边缘设备上直接训练深度神经网络(DNN)正受到越来越多关注,因其能有效应对领域适配和隐私保护等挑战。然而,传统DNN训练依赖大规模数据集,给边缘设备带来巨大负担,尤其对新兴的大语言模型(LLM)任务更为显著。为此,已有无需DNN的采样方法NMS被提出:先对数据集进行降维,再在低维空间中选取代表性样本,避免了基于DNN方法的结构偏见,从而获得更好泛化性能。但现有最优方法NMS存在两大缺陷:(1)搜索方法与困惑度误差函数的非单调特性不匹配,导致降维表示中出现异常点;(2)目标困惑度等关键参数依赖经验设定,引入主观性并造成采样不均。这两点共同引发样本代表性偏差,导致训练精度下降。为此,本文提出AdapSNE,融合高效的非单调搜索方法——烟花算法(FWA),以抑制异常点,并采用熵引导优化实现均匀采样,确保训练样本的代表性,进而提升训练准确率。为降低FWA搜索与熵优化带来的边缘侧迭代计算开销,设计了具有定制数据流和时分复用机制的加速器,显著减少设备端训练的能量消耗与硬件面积占用。

原文摘要 · Abstract (English)

Training deep neural networks (DNNs) directly on edge devices has attracted increasing attention, as it offers promising solutions to challenges such as domain adaptation and privacy preservation. However, conventional DNN training typically requires large-scale datasets, which imposes prohibitive overhead on edge devices-particularly for emerging large language model (LLM) tasks. To address this challenge, a DNN-free method (ie., dataset sampling without DNN), named NMS (Near-Memory Sampling), has been introduced. By first conducting dimensionality reduction of the dataset and then performing exemplar sampling in the reduced space, NMS avoids the architectural bias inherent in DNN-based methods and thus achieves better generalization. However, The state-of-the-art, NMS, suffers from two limitations: (1) The mismatch between the search method and the non-monotonic property of the perplexity error function leads to the emergence of outliers in the reduced representation; (2) Key parameter (ie., target perplexity) is selected empirically, introducing arbitrariness and leading to uneven sampling. These two issues lead to representative bias of examplars, resulting in degraded accuracy. To address these issues, we propose AdapSNE, which integrates an efficient non-monotonic search method-namely, the Fireworks Algorithm (FWA)-to suppress outliers, and employs entropy-guided optimization to enforce uniform sampling, thereby ensuring representative training samples and consequently boosting training accuracy. To cut the edge-side cost arising from the iterative computations of FWA search and entropy-guided optimization, we design an accelerator with custom dataflow and time-multiplexing markedly reducing on-device training energy and area.

边缘计算数据采样优化算法模型压缩

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。