提出动态加权的随机重排算法，显著提升大规模优化的可扩展性。
Adjusted Shuffling SARAH: Advancing Complexity Analysis via Dynamic Gradient Weighting
- 通过动态权重融合重排策略，改进递归SARAH框架的探索能力。
- 无须依赖数据集规模的复杂度，大幅降低大规模场景下的计算开销。
- 适合追求理论最优与高可扩展性的大规模机器学习优化任务。
本文提出调整后的随机重排SARAH（Adjusted Shuffling SARAH），将重排策略结合递归SARAH框架，并引入动态加权机制以增强探索能力。我们分析了两种运行模式：首先，在精确模式下，该算法在强凸和非凸设置中均达到现有重排方差缩减方法的最佳理论性能；其次，为应对大规模场景，我们引入近似模式，采用小批量估计器。本工作的关键贡献在于证明该近似模式的总复杂度与数据集大小无关，因此在样本量较大时，相比现有重排方法具有显著更高的可扩展性。
原文摘要 · Abstract (English)
In this paper, we propose Adjusted Shuffling SARAH, a novel algorithm that integrates shuffling strategies into the recursive SARAH framework using a dynamic weighting mechanism to enhance exploration. We analyze the algorithm under two operating modes. First, we show that the Exact Mode matches the best-known theoretical guarantees for shuffling variance-reduced methods in both strongly convex and non-convex settings. Second, to address large-scale regimes, we introduce an Inexact Mode that utilizes mini-batch estimators. A key contribution of our work is proving that this Inexact Mode achieves a total complexity independent of the dataset size, making it significantly more scalable than existing shuffling methods when the sample size is large.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。