arXiv:2603.20296cs.LGcs.AI2026-03中稿 · IEEE ICME 2026被引 1

让边缘设备逐步学习复杂视觉知识,提升分布式图像识别效果。

Collaborative Adaptive Curriculum for Progressive Knowledge Distillation

  • 按知识复杂度分层传递,动态调整客户端学习进度。
  • 在CIFAR-10上比FedAvg高3.64%准确率,收敛速度提升一倍。
  • 适合资源受限、数据差异大的边缘视觉分析场景。

近期协同知识蒸馏进展在资源受限的分布式多媒体学习中展现出顶尖性能,但其应用面临核心矛盾:教师模型知识维度高与客户端能力异构之间的不匹配,阻碍了在边缘视觉分析系统中的部署。受课程学习启发,本文提出联邦自适应渐进蒸馏(FAPD),一种以共识驱动的自适应知识迁移框架。FAPD通过基于PCA的结构化方法,将教师特征分层分解,提取按方差贡献排序的主成分,构建自然的视觉知识层级。客户端通过维度自适应投影矩阵逐步接收更高复杂度的知识。同时,服务器通过跟踪全局准确率在时间共识窗口内的波动,仅在集体共识形成时才推进课程复杂度。实验结果表明,该方法可证明地调节知识传递节奏,在三个数据集上均表现优异:在CIFAR-10上相较FedAvg提升3.64%准确率,收敛速度提升2倍,并在极端数据异构(α=0.1)下仍保持鲁棒性,优于基线超过4.5%。

原文摘要 · Abstract (English)

Recent advances in collaborative knowledge distillation have demonstrated cutting-edge performance for resource-constrained distributed multimedia learning scenarios. However, achieving such competitiveness requires addressing a fundamental mismatch: high-dimensional teacher knowledge complexity versus heterogeneous client learning capacities, which currently prohibits deployment in edge-based visual analytics systems. Drawing inspiration from curriculum learning principles, we introduce Federated Adaptive Progressive Distillation (FAPD), a consensus-driven framework that orchestrates adaptive knowledge transfer. FAPD hierarchically decomposes teacher features via PCA-based structuring, extracting principal components ordered by variance contribution to establish a natural visual knowledge hierarchy. Clients progressively receive knowledge of increasing complexity through dimension-adaptive projection matrices. Meanwhile, the server monitors network-wide learning stability by tracking global accuracy fluctuations across a temporal consensus window, advancing curriculum dimensionality only when collective consensus emerges. Consequently, FAPD provably adapts knowledge transfer pace while achieving superior convergence over fixed-complexity approaches. Extensive experiments on three datasets validate FAPD's effectiveness: it attains 3.64% accuracy improvement over FedAvg on CIFAR-10, demonstrates 2x faster convergence, and maintains robust performance under extreme data heterogeneity (α=0.1), outperforming baselines by over 4.5%.

知识蒸馏联邦学习边缘计算自适应

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。