提出首个分布式鲁棒核心集选择方法，解决边缘计算中的高效隐私保护学习问题。
First-order Constrained Trilevel Optimization Over Distributed Networks for Robust Coreset Selection
- 构建三层次优化框架，融合核心集选择、鲁棒优化与分布式学习的层级依赖
- 设计F²CTO算法，实现分布式求解并达到O(ε⁻³ᐟ²)非渐近收敛速度
- 适用于物联网等隐私敏感场景下的持续学习，兼顾效率与鲁棒性
随着物联网快速发展，海量数据在分布式边缘网络中产生。全量数据训练带来显著计算开销和存储瓶颈，使核心集选择成为关键范式。鉴于本地数据的隐私敏感性及实际部署中对模型鲁棒性的高要求，开发有效的分布式优化框架以实现鲁棒核心集选择至关重要，但尚未被充分探索。本文首次刻画了核心集选择、鲁棒优化与分布式学习之间的层级依赖关系，将分布式鲁棒核心集选择建模为带层级约束的三层次优化问题。为此，提出联邦一阶约束三层次优化（F²CTO）方法，通过分层复合值函数重构与分布式交替投影梯度算法协同求解。据我们所知，F²CTO是首个用于分布式鲁棒核心集选择的方法，也是首个针对具有层级约束的三层次优化问题的分布式求解方法。理论证明该方法在寻找ε-驻点时具有O(ε⁻³ᐟ²)的非渐近收敛率。在可靠持续学习任务上的大量实验验证了F²CTO的有效性与高效性。
原文摘要 · Abstract (English)
With the rapid advancement of the Internet of Things (IoT), massive amounts of data are generated across distributed edge networks. Training models on full data incurs significant computational overhead and storage bottlenecks, rendering coreset selection a critical paradigm. Furthermore, given the privacy-sensitive nature of local data and the escalating demand for model robustness in real-world deployments, developing an effective distributed optimization framework for robust coreset selection is vital, yet remains largely unexplored. To this end, this work first characterizes the hierarchical dependencies among coreset selection, robust optimization, and distributed learning, and formulates the distributed robust coreset selection as a trilevel optimization problem with level-wise constraints. Furthermore, to effectively solve the trilevel problem in a distributed manner, the \underline{F}ederated \underline{F}irst-order \underline{C}onstrained \underline{T}rilevel \underline{O}ptimization (F$^2$CTO) is proposed, which synergistically integrates a hierarchical composite value-function reformulation and a distributed alternating projected gradient algorithm. To the best of our knowledge, F$^2$CTO is the first method developed for distributed robust coreset selection, as well as the first distributed optimization approach for trilevel optimization problems with level-wise constraints. Additionally, we prove that the proposed method achieves a non-asymptotic convergence rate of $\mathcal{O}(ε^{-3/2})$ for finding an $ε$-stationary point. Extensive empirical evaluations on reliable continual learning demonstrate the effectiveness and efficiency of the proposed F$^2$CTO.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。