arXiv:2606.19734cs.LG2026-06中稿 · ICML

考虑决策影响数据分布的联邦双层优化,提升模型稳定性与泛化能力。

Federated Bilevel Performative Prediction

  • 从决策依赖分布出发,构建联邦双层可执行稳定点新范式。
  • 提出两种方法,线性收敛与通信高效,适用于非凸神经网络场景。
  • 在策略回归与元分类任务中验证稳定性阈值,优于传统基线。

联邦双层优化广泛应用于分布式客户端的嵌套学习问题,如隐私与通信受限下的联邦超参数调优和元学习。现有方法多假设客户端数据分布固定,但实际中部署决策会改变客户端行为与数据收集,导致客户端特异、决策依赖的分布漂移。本文研究联邦双层可执行预测,其中上下层目标均在客户端相关、决策相关的分布下评估。从解耦风险视角形式化了联邦双层可执行稳定(FBPS)点,并给出了其存在性与唯一性的充分条件。进一步提出两种联邦算法计算FBPS解:FBi-RRM在收缩条件下实现线性收敛;FBi-SGD基于联邦超梯度估计,通信高效,当敏感度足够小时,在递减步长下具备收敛保证。在策略回归与元策略分类实验中,验证了预测的稳定阈值,展示了优于非可执行基线的元泛化性能;基于CNN的分类任务进一步证明了所提方法在非凸神经网络设置下的实际有效性。

原文摘要 · Abstract (English)

Federated bilevel optimization is widely used for nested learning problems across distributed clients, such as federated hyperparameter tuning and meta-learning under privacy and communication constraints. Most existing formulations assume fixed client data distributions, which can be violated by performativity, where deployed decisions reshape client behavior and data collection, inducing client-specific, decision-dependent distribution shift. We study federated bilevel performative prediction, where both upper-level (UL) and lower-level (LL) objectives are evaluated under client-dependent, decision-dependent distributions. We formalize the federated bilevel performatively stable (FBPS) point under a decoupled-risk perspective and provide sufficient conditions for its existence and uniqueness. We then develop two federated methods to compute the FBPS solution: FBi-RRM, which converges linearly under a contraction condition, and FBi-SGD, a communication-efficient stochastic method based on federated hypergradient estimation with convergence guarantees under diminishing step sizes when sensitivities are sufficiently small. Experiments on strategic regression and meta strategic classification validate the predicted stability thresholds and demonstrate improved meta-generalization over non-performative baselines, and CNN-based classification further demonstrates the practical effectiveness of the proposed methods in nonconvex neural network settings.

联邦学习双层优化可执行预测元学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。