用混合图像和噪声图缓解联邦学习中的标签不平衡问题。
Balancing Label Imbalance in Federated Environments Using Only Mixup and Artificially-Labeled Noise
- 结合真实数据与混合法生成伪图像，平衡客户端标签分布。
- 少量混合图像和自然噪声即可显著提升CIFAR-10和MNIST的训练效果。
- 无需额外训练，利用未训练的StyleGAN生成噪声图，适合资源受限场景。
在分布式或联邦学习环境中，各客户端的数据常偏向不同标签子集，即非独立同分布（non-iid）情况，严重影响模型训练与性能。本文探索一种简单而有效的增强策略：通过伪图像填补某一标签类别的不足样本。不同于以往仅使用本地数据混合法生成伪图像的算法，本文在客户端数据集中同时包含真实数据与伪图像。此外，采用DP-Instahide变体降低图像编码可解码性，并引入无训练、基于未训练StyleGAN生成的“自然噪声”作为补充数据。这些噪声图像模拟自然场景的功率谱特征，与混合法图像共同促进客户端间标签分布的均质化。实验表明，少量混合法与自然噪声增强即可显著改善标签偏斜的CIFAR-10与MNIST训练表现。
原文摘要 · Abstract (English)
Clients in a distributed or federated environment will often hold data skewed towards differing subsets of labels. This scenario, referred to as heterogeneous or non-iid federated learning, has been shown to significantly hinder model training and performance. In this work, we explore the limits of a simple yet effective augmentation strategy for balancing skewed label distributions: filling in underrepresented samples of a particular label class using pseudo-images. While existing algorithms exclusively train on pseudo-images such as mixups of local training data, our augmented client datasets consist of both real and pseudo-images. In further contrast to other literature, we (1) use a DP-Instahide variant to reduce the decodability of our image encodings and (2) as a twist, supplement local data using artificially labeled, training-free 'natural noise' generated by an untrained StyleGAN. These noisy images mimic the power spectra patterns present in natural scenes which, together with mixup images, help homogenize label distribution among clients. We demonstrate that small amounts of augmentation via mixups and natural noise markedly improve label-skewed CIFAR-10 and MNIST training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。