arXiv:2604.01030cs.CV2026-04被引 1

让3D高斯点云快速生成更准,通过可微优化预测初始参数。

Diff3R: Feed-forward 3D Gaussian Splatting with Uncertainty-aware Differentiable Optimization

  • 用可微优化层学习最优初始化,替代传统零样本输出。
  • 结合隐函数定理与矩阵无关求解器,降低反向传播开销。
  • 引入不确定性建模,提升对噪声和稀疏输入的鲁棒性。

3D高斯点云渲染(3DGS)近年发展出两种方向:前馈模型在少视角下推理速度快,而逐场景优化虽质量高但计算成本大。为融合两者优势,我们提出Diff3R,将可微3DGS优化层直接嵌入训练过程,使网络学会预测测试时优化的最佳初始状态,而非传统零样本结果。为解决优化步骤反向传播的开销,提出基于隐函数定理与专用于3DGS的矩阵无关共轭梯度求解器。此外,通过数据驱动的不确定性模型,自适应控制优化中参数变化幅度,有效缓解欠约束区域过拟合,增强对输入异常值的鲁棒性。该优化层具有模型无关性,可无缝集成至现有前馈3DGS架构,适用于有位姿输入和无位姿输入的方法,显著提升测试时优化效果。

原文摘要 · Abstract (English)

Recent advances in 3D Gaussian Splatting (3DGS) present two main directions: feed-forward models offer fast inference in sparse-view settings, while per-scene optimization yields high-quality renderings but is computationally expensive. To combine the benefits of both, we introduce Diff3R, a novel framework that explicitly bridges feed-forward prediction and test-time optimization. By incorporating a differentiable 3DGS optimization layer directly into the training loop, our network learns to predict an optimal initialization for test-time optimization rather than a conventional zero-shot result. To overcome the computational cost of backpropagating through the optimization steps, we propose computing gradients via the Implicit Function Theorem and a scalable, matrix-free PCG solver tailored for 3DGS optimization. Additionally, we incorporate a data-driven uncertainty model into the optimization process by adaptively controlling how much the parameters are allowed to change during optimization. This approach effectively mitigates overfitting in under-constrained regions and increases robustness against input outliers. Since our proposed optimization layer is model-agnostic, we show that it can be seamlessly integrated into existing feed-forward 3DGS architectures for both pose-given and pose-free methods, providing improvements for test-time optimization.

3D生成高斯点云可微优化实时渲染

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。