提出抗干扰的异步优化方法，提升分布式学习鲁棒性与效率。
Adversary-Robust Learning from Fully Asynchronous Directional Derivative Estimates
- 基于符号方向投影的异步更新机制，降低恶意节点影响。
- 理论证明几乎必然收敛至驻点，一阶与零阶场景分别达到近最优率。
- 无需服务器私有数据集，实验显示精度和速度均优于传统方法。
我们提出FAR-SIGN（通过符号方向投影实现完全异步鲁棒优化），用于参数服务器-工作节点系统中的抗干扰学习。FAR-SIGN通过沿精心设计方向的符号更新实现鲁棒性，并利用双时标机制缓解由此带来的偏差。该方法支持一阶与零阶实现，可在无需服务器私有参考数据集的情况下实现完全异步执行。我们证明了FAR-SIGN在光滑非凸目标下几乎必然收敛至驻点集合。此外，在一阶设置中，其收敛速率达到近最优的$O(n^{-1/4+ε})$，在零阶设置中为标准的$O(n^{-1/6+ε})$，其中 $n$ 为迭代次数，$ε>0$ 可任意小。在MNIST上的实验表明，FAR-SIGN在准确率和实际运行时间上均优于基于鲁棒聚合的方法。
原文摘要 · Abstract (English)
We propose FAR-SIGN (Fully Asynchronous Robust optimization via SIGNed directional projections) for adversary-resilient learning in parameter-server--worker systems. FAR-SIGN achieves robustness through sign-based updates along carefully designed directions and mitigates the resulting bias via a two-timescale mechanism. It admits both first-order and zeroth-order implementations and enables fully asynchronous execution without requiring a private reference dataset at the server. We establish almost-sure convergence of FAR-SIGN to the set of stationary points for smooth, nonconvex objectives. Moreover, we prove the near-optimal rate of $O(n^{-1/4+ε})$ in the first-order setting and the standard $O(n^{-1/6+ε})$ in the zeroth-order setting, where $n$ is the iteration count and $ε>0$ can be chosen arbitrarily small. Experiments on MNIST show that FAR-SIGN outperforms robust aggregation-based methods in both accuracy and wall-clock time.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。