提出FSGD，让SGD在高维数据中边流式处理边学习潜在因子。
Factor Augmented High-Dimensional SGD
- 用潜在因子增强的SGD，直接处理流式数据，无需预存全部数据。
- 首次将因子估计误差纳入理论分析，证明了小步长下的收敛性。
- 适合大规模高维机器学习系统，尤其对实时更新场景友好。
随机梯度下降（SGD）是现代机器学习中的基础优化算法。本文提出因子增强型SGD（FSGD），一种在高维学习任务中利用潜在因子表示的新优化方法。与依赖离线表示学习和全量数据存储的标准两阶段降维方法不同，FSGD的核心创新在于仅需处理流式数据，可扩展至大规模高维问题。此外，我们建立了首个将潜在因子估计误差显式纳入分析的理论框架，在衰减步长与小批量更新下，证明了ℓ^s范数下的矩收敛性。该结果为在高维机器学习系统中可靠、可扩展地使用SGD提供了新基础。
原文摘要 · Abstract (English)
Stochastic gradient descent (SGD) is a fundamental optimization algorithm widely used in modern machine learning. In this paper, we propose Factor-Augmented SGD (FSGD), a new optimization method that leverages latent factor representations in high-dimensional learning tasks. Unlike standard two-stage dimension reduction approaches that rely on offline representation learning and full data storage, a key novelty of FSGD is that it operates purely on streaming data, making it scalable to large-scale and high-dimensional problems. Furthermore, we establish the first theoretical framework that explicitly incorporates latent factor estimation error into the analysis of SGD, and provide moment convergence in $\ell^s$ norm under decaying step sizes and mini-batch updates. Our results provide a new foundation for employing SGD reliably and scalably in high-dimensional machine learning systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。