L-SR1用学习法改进二阶优化,加速收敛且无需额外标注数据。
L-SR1: Learned Symmetric-Rank-One Preconditioning
- 基于经典SR1方法设计可学习的对称秩一预条件器。
- 在人体网格恢复任务中减少迭代次数,优于传统与现有学习优化器。
- 模型小巧、泛化强,适合各类需快速收敛的优化场景。
端到端深度学习虽成果显著,但依赖大量标注数据,泛化能力有限,计算开销大。传统优化方法数据效率高、轻量,却常收敛缓慢。学习型优化器试图弥合差距,但多数聚焦一阶方法,二阶学习优化研究不足。本文提出L-SR1,一种受经典对称秩一(SR1)方法启发的可学习二阶优化器。其核心是投影引导的割线机制(PGSM),生成半正定预条件矩阵,并引导元训练符合拟牛顿割线关系。通过可控解析基准测试,研究了稳定性、跨问题维度的泛化性及搜索方向质量,并在单目人体网格恢复(HMR)任务上评估,表现优于经典及学习型优化基线。模型紧凑,无需任务特异性微调或标注数据,展现出强泛化能力,可广泛集成于各类迭代优化问题以加速收敛、减少迭代次数。
原文摘要 · Abstract (English)
End-to-end deep learning has achieved impressive results but often relies on large labeled datasets, exhibits limited generalization to unseen scenarios, and incurs substantial computational cost. Classical optimization methods, in contrast, are more data-efficient and lightweight but frequently suffer from slow convergence. Learned optimizers aim to bridge this gap, yet existing approaches have focused primarily on first-order methods, while learned second-order optimization has received much less attention. We introduce L-SR1, a learned second-order optimizer inspired by the classical Symmetric Rank-One (SR1) method. At its core, L-SR1 employs a Projection-Guided Secant Mechanism (PGSM) that generates positive semi-definite preconditioners and biases meta-training toward the quasi-Newton secant relation. Through controlled analytic benchmarks, we study stability, generalization across problem dimensions, and search direction quality, and further evaluate L-SR1 on Monocular Human Mesh Recovery (HMR), where it outperforms both classical and learned optimization-based baselines. With a compact model and no reliance on task-specific fine-tuning or annotated data, L-SR1 demonstrates strong generalization and can be integrated into a broad range of iterative optimization problems to accelerate convergence and reduce the required number of iterations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。