arXiv:2608.07157cs.LGcs.DC2026-08

提出自适应子模型联邦学习的容量混淆问题,揭示分配策略失效根源。

Capacity Confounds and Coverage Guarantees in Adaptive Sub-model Federated Learning

  • 基于客户端更新差异估计异构性,结果受设备容量主导而非真实数据差异。
  • 当所有客户端容量受限时,未覆盖参数随机初始化并污染全局模型。
  • 性能提升源于容量预算与参数覆盖,非智能分配策略本身。

子模型联邦学习允许资源受限客户端训练全局模型的宽度缩减版本,但现有方法仅依据设备资源分配容量。一种更优思路是根据客户端数据异构性动态分配,该异构性可通过服务器观测到的更新估计。本文以自适应容量分配框架HAS-FL为案例进行验证。首先,在可复现的数据划分下,基于更新差异的异构性估计被发现主要受设备容量影响:在多个数据集、所有随机种子和两种修正估计器中,估计值与容量呈强负相关,控制容量后已无数据信号残留。这一未被记录的混淆效应影响所有基于子模型更新估计客户端统计的方法。其次,自适应分配存在隐含失败模式:当所有客户端容量低于全宽时,未覆盖参数保持随机初始化状态,逐步污染全局模型;引入简单的覆盖保证可消除该失败,并解释为何均匀分配会失效。第三,通过同预算对照实验发现:随机分配与平均预算相同,在图像基准上表现相当;在自然划分的文本基准上,自适应策略反而是三者中最差,且消耗最多容量。子模型训练仍具价值,因其将成本降至二次级,但准确率保护机制实为参数覆盖,而非分配智能。其表面优势来自容量预算与覆盖机制,未来设计需分离异构性信号与容量影响。

原文摘要 · Abstract (English)

Sub-model federated learning lets resource-constrained clients train width-reduced versions of a global model, but existing methods allocate capacity by device resources alone. A natural next step, allocating capacity by each client's data heterogeneity as estimated from the updates the server already observes, has been repeatedly suggested. We ask whether that step is possible, using HAS-FL, an adaptive capacity-allocation framework, as a test case. Our findings are threefold. First, validated against ground-truth label-distribution divergence on reproducible partitions, update-divergence estimates of client heterogeneity are dominated by capacity rather than data: across two corrected estimators, multiple datasets, and all seeds, the estimates correlate strongly and negatively with device capacity, and no data signal remains once capacity is controlled for. This previously undocumented confound affects any method estimating client statistics from sub-model updates. Second, adaptive allocation has a hidden failure mode: when every client is capped below full width, the uncovered parameters stay at random initialization and progressively corrupt the global model. A simple coverage guarantee removes the failure and explains why uniform allocation collapses. Third, a matched-budget control settles what adaptivity contributes: random allocation to the same average budget performs no differently on both image benchmarks, and on the naturally partitioned text benchmark the adaptive policy is the weakest of the three strategies while consuming the most capacity. Sub-model training remains valuable because it admits constrained clients at quadratically reduced cost, but what protects accuracy is parameter coverage rather than allocation intelligence. Its apparent benefits come from capacity budgeting and coverage, and future designs need heterogeneity signals separable from capacity effects.

联邦学习容量分配异构性参数覆盖

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。