边端设备联合训练时动态筛选关键特征,兼顾精度与资源开销。
Embedded Federated Feature Selection with Dynamic Sparse Training: Balancing Accuracy-Cost Tradeoffs
- 训练中同步动态剪枝与再生神经元,实时剔除无效特征。
- 在9个高维数据集上实现更优的精度-成本平衡,通信量减少30%以上。
- 适合资源受限的边缘计算场景,如医疗、语音和图像分析。
联邦学习(FL)允许多个资源受限的边缘设备在异构条件下协同训练全局模型。然而,低算力设备会成为瓶颈,拖慢收敛速度。现有联邦特征选择(FFS)方法通常独立于联邦学习或依赖第三方,增加计算与通信开销,难以应用于真实世界的高维数据。为此,本文提出首个嵌入式动态稀疏联邦特征选择(DSFFS),在训练过程中同步进行特征选择。通过动态剪枝与再生输入层神经元及其连接、隐藏层连接,自动剔除非信息特征。该方法显著提升设备端计算效率,降低网络通信负担,并增强全局模型性能。在涵盖生物、图像、语音、文本等领域的9个真实世界高维数据集上,基于非独立同分布(non-iid)数据设置的实验表明,相比现有最优方法,本方法在保持更高准确率的同时,有效降低了通信与计算成本。
原文摘要 · Abstract (English)
Federated Learning (FL) enables multiple resource-constrained edge devices with varying levels of heterogeneity to collaboratively train a global model. However, devices with limited capacity can create bottlenecks and slow down model convergence. One effective approach to addressing this issue is to use an efficient feature selection method, which reduces overall resource demands by minimizing communication and computation costs, thereby mitigating the impact of struggling nodes. Existing federated feature selection (FFS) methods are either considered as a separate step from FL or rely on a third party. These approaches increase computation and communication overhead, making them impractical for real-world high-dimensional datasets. To address this, we present \textit{Dynamic Sparse Federated Feature Selection} (DSFFS), the first innovative embedded FFS that is efficient in both communication and computation. In the proposed method, feature selection occurs simultaneously with model training. During training, input-layer neurons, their connections, and hidden-layer connections are dynamically pruned and regrown, eliminating uninformative features. This process enhances computational efficiency on devices, improves network communication efficiency, and boosts global model performance. Several experiments are conducted on nine real-world datasets of varying dimensionality from diverse domains, including biology, image, speech, and text. The results under a realistic non-iid data distribution setting show that our approach achieves a better trade-off between accuracy, computation, and communication costs by selecting more informative features compared to other state-of-the-art FFS methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。