arXiv:2605.17552cs.LG2026-05

针对边缘联邦学习内存瓶颈,提出分块量化优化器,显著降低内存占用且不损失精度。

Q-LocalAdam: Memory-Efficient Client-Side Adaptive Optimization for Edge Federated Learning

论文配图:Q-LocalAdam: Memory-Efficient Client-Side Adaptive Optimization for Edge Federated Learning
图 1 · 摘自论文原文
  • 根据动量和方差分布特性,分别采用8位线性编码和对数空间编码进行压缩
  • 在非独立同分布数据下实现3.37倍内存减少,极端异构时准确率提升5.74个百分点
  • 适用于资源受限的边缘设备,支持更大模型和更多并行任务

边缘设备上的联邦学习面临数据非独立同分布和内存预算紧张的双重挑战。自适应优化器如Adam虽能稳定训练,但需存储全精度动量与方差状态,通常使客户端内存开销增加三倍,限制了可部署模型规模及并发联邦任务数量。我们实证发现,联邦Adam中的动量值具有对称有界特性,而方差跨度达八数量级,呈对数正态分布。基于此不对称性,提出Q-LocalAdam:对动量采用分布感知的8位分块线性量化编码,对方差采用对数空间编码,同时保持模型参数为全精度。在CIFAR-10与CIFAR-100上,于不同数据异构度(α∈{0.1, 0.5, 1.0, IID})下,该方法实现3.37倍优化器内存压缩,中等异构下无精度损失,极端异构下显著提效(如CIFAR-100,α=0.1时提升5.74个百分点)。多种子验证确认统计显著性(p<0.01)。相比之下,朴素均匀量化退化至随机性能,表明分布感知设计至关重要。Q-LocalAdam可在不修改联邦协议前提下,提升边缘设备上大模型与多任务并发能力。

原文摘要 · Abstract (English)

Federated learning on edge devices must cope with non-IID client data and tight memory budgets. Adaptive optimizers like Adam stabilize training under data heterogeneity but require storing full-precision momentum and variance states, often tripling client memory overhead. This limits deployable model sizes and concurrent federated jobs on resource-constrained devices. We empirically observe that momentum and variance in federated Adam exhibit fundamentally different statistical properties: momentum values are symmetric and bounded, while variance spans eight orders of magnitude with log-normal structure. Motivated by this asymmetry, we propose \textbf{Q-LocalAdam}, which applies distribution-aware 8-bit quantization block-wise linear encoding for momentum and log-space encoding for variance while keeping model parameters in full precision. Across CIFAR-10 and CIFAR-100 under varying data heterogeneity ($α\in \{0.1, 0.5, 1.0, \text{IID}\}$), Q-LocalAdam achieves $3.37\times$ optimizer memory reduction with no accuracy loss under moderate heterogeneity and significant improvements under extreme heterogeneity (e.g., +5.74pp on CIFAR-100, $α=0.1$). Multi-seed validation confirms statistical significance ($p<0.01$). In contrast, naive uniform quantization degrades to random performance, demonstrating that distribution-aware design is essential. Q-LocalAdam enables larger models and more concurrent workloads on memory-constrained edge devices without modifying the federated protocol.

联邦学习内存优化边缘计算量化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。