动态调整分数阶梯度，让优化更快更稳。
Effective Dimension Aware Fractional-Order Stochastic Gradient Descent for Convex Optimization Problems
- 根据模型敏感性和有效维度自动调节分数阶指数
- 在噪声环境下收敛速度提升，参数估计更鲁棒
- 适合需要稳定优化的复杂建模任务
分数阶随机梯度下降（FOSGD）利用分数阶指数捕捉优化中的长记忆效应，但其应用常受限于指数调参困难和不稳定性。本文提出2SED分数阶随机梯度下降（2SEDFOSGD），将双尺度有效维度（2SED）算法与FOSGD结合，实现数据驱动的分数阶指数自适应。通过追踪模型敏感性和有效维度，2SEDFOSGD动态调节指数以抑制振荡、加速收敛。理论上，该方法保留了分数阶记忆优势，避免了原始方法中迟缓或不稳定的缺陷。在自回归（AR）模型的高斯噪声和α-稳定噪声场景下，以及在MNIST和CIFAR-100图像分类任务上的实验表明，该方法具有更快的收敛速度和更鲁棒的参数估计，凸显了维度感知分数阶技术在高级建模与估计任务中的潜力。
原文摘要 · Abstract (English)
Fractional-order stochastic gradient descent (FOSGD) leverages fractional exponents to capture long-memory effects in optimization. However, its utility is often limited by the difficulty of tuning and stabilizing these exponents. We propose 2SED Fractional-Order Stochastic Gradient Descent (2SEDFOSGD), which integrates the Two-Scale Effective Dimension (2SED) algorithm with FOSGD to adapt the fractional exponent in a data-driven manner. By tracking model sensitivity and effective dimensionality, 2SEDFOSGD dynamically modulates the exponent to mitigate oscillations and hasten convergence. Theoretically, this approach preserves the advantages of fractional memory without the sluggish or unstable behavior observed in naïve fractional SGD. Empirical evaluations in Gaussian and $α$-stable noise scenarios using an autoregressive (AR) model\textcolor{red}{, as well as on the MNIST and CIFAR-100 datasets for image classification,} highlight faster convergence and more robust parameter estimates compared to baseline methods, underscoring the potential of dimension-aware fractional techniques for advanced modeling and estimation tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。