解决垂直联邦学习中特征缺失问题,支持不完整数据训练与推理。
Vertical Federated Learning with Missing Features During Training and Inference
- 通过参数共享与任务采样构建多预测器家族应对任意特征缺失。
- 在特征缺失概率0.5时,CIFAR-100上准确率提升19.3%。
- 适用于实际场景中客户端随时退出或数据不全的情况。
垂直联邦学习在多个客户端间基于特征划分的数据集上协同训练模型,无需共享本地数据。传统方法假设训练和推理时所有特征分区均可用,但现实中常因部分客户端未观测到其特征而无法满足。忽略不完整样本会损害泛化能力,而推理时无法支持也限制模型实用性。若某客户端训练后退出,其特征块将不可用,导致模型失效。为此,我们提出LASER-VFL,一种可处理任意特征分区集合的垂直联邦学习方法,用于高效训练与推理分块神经网络。该方法依赖模型参数共享与任务采样,训练一组预测器。理论证明,对于非凸目标,收敛速率达$/mathcal{O}({1}/{ ext{sqrt}{T}})$;在Polyak-Łojasiewicz条件下,收敛至最优解邻域为线性。数值实验显示,相比基线,即使无缺失特征时,性能亦有提升:在四个特征块各以0.5概率观测下,CIFAR-100准确率提升19.3%;全部特征可观测时,提升9.5%。代码已开源。
原文摘要 · Abstract (English)
Vertical federated learning trains models from feature-partitioned datasets across multiple clients, who collaborate without sharing their local data. Standard approaches assume that all feature partitions are available during both training and inference. Yet, in practice, this assumption rarely holds, as for many samples only a subset of the clients observe their partition. However, not utilizing incomplete samples during training harms generalization, and not supporting them during inference limits the utility of the model. Moreover, if any client leaves the federation after training, its partition becomes unavailable, rendering the learned model unusable. Missing feature blocks are therefore a key challenge limiting the applicability of vertical federated learning in real-world scenarios. To address this, we propose LASER-VFL, a vertical federated learning method for efficient training and inference of split neural network-based models that is capable of handling arbitrary sets of partitions. Our approach is simple yet effective, relying on the sharing of model parameters and on task-sampling to train a family of predictors. We show that LASER-VFL achieves a $\mathcal{O}({1}/{\sqrt{T}})$ convergence rate for nonconvex objectives and, under the Polyak-Łojasiewicz inequality, it achieves linear convergence to a neighborhood of the optimum. Numerical experiments show improved performance of LASER-VFL over the baselines. Remarkably, this is the case even in the absence of missing features. For example, for CIFAR-100, we see an improvement in accuracy of $19.3\%$ when each of four feature blocks is observed with a probability of 0.5 and of $9.5\%$ when all features are observed. The code for this work is available at https://github.com/Valdeira/LASER-VFL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。