提出可长期稳定优化3D高斯点云的新学习型优化器,无需调参。
Learn2Splat: Extending the Horizon of Learned 3DGS Optimization

- 用元学习+检查点缓存扩展优化时长,让更新更连贯。
- 在长周期优化中保持稳定,早期新视角重建质量更好。
- 首次统一评测学习与传统优化器,适合3D重建研究者。
3D高斯点云(3DGS)优化通常采用标准优化器(如Adam、SGD),虽对多种场景稳定,但为通用设计,未考虑场景内部结构与空间关系,导致参数更新独立,效率低、收敛慢。近期方法引入学习型优化器,通过捕捉参数间依赖关系实现相关更新,但需固定迭代次数训练,且依赖人工调度学习率以避免性能退化。本文提出一种新型学习型优化器,可在不依赖辅助机制的前提下实现长期稳定优化。核心是结合检查点缓冲与优化器回放策略的元学习框架,并设计能编码梯度尺度信息的隐状态架构。实验表明,该方法在长优化周期中仍保持稳定,早期新视角生成质量显著提升,且具备零样本泛化能力,适用于未见过的重建设置。为此,我们构建首个统一训练与评估学习型与传统优化器的框架,覆盖稀疏与密集视图场景。代码与模型将公开发布。
原文摘要 · Abstract (English)
3D Gaussian Splatting (3DGS) optimization is most commonly performed using standard optimizers (Adam, SGD). While stable across diverse scenes, standard optimizers are general-purpose and not tailored to the structure of the problem. In particular, they produce independent parameter updates that do not capture the structural and spatial relationships within a scene, leading to inefficient optimization and slow convergence. Recent works introduced learned optimizers that predict correlated updates informed by inter-parameter and inter-Gaussian dependencies. However, these methods are trained for a fixed number of optimization iterations and rely on manually scheduled learning rates to avoid degradation. In this paper, we introduce a learned optimizer for 3DGS that avoids degradation over extended optimization horizons without auxiliary mechanisms. To enable this, we propose a meta-learning scheme that extends the optimization horizon via a checkpoint buffer and an optimizer rollout strategy, combined with an architecture that encodes gradient scale information in its latent states. Results show improved early novel view synthesis quality while remaining stable over long horizons, with zero-shot generalization to unseen reconstruction settings. To support our findings, we introduce the first unified framework for training and evaluating both learned and conventional optimizers across sparse and dense view settings. Code and models will be released publicly. Our project page is available at https://naamapearl.github.io/learn2splat .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。