边拍边建模，实现3D高斯点云的秒级实时优化。
Gaussian On-the-Fly Splatting: A Progressive Framework for Robust Near Real-Time 3DGS Optimization
- 新图像到来时即时更新位姿与稀疏点，同步优化并融合新高斯点。
- 每张新图优化仅需数秒，渲染损失极小，训练效率大幅提升。
- 适合需要快速重建的移动场景应用，如AR/VR与实时导航。
3D高斯点云（3DGS）能实现高质量实时渲染，但现有方法依赖完整结构光（SfM）处理后的离线训练。本文提出「即插即用3DGS」（On-the-Fly GS），一种在图像采集过程中实现近实时3DGS优化的渐进式框架。每帧图像到达后，通过即插即用SfM更新其位姿与稀疏点，并将新优化的高斯点立即融入3DGS场。为此，我们设计了局部与半全局渐进优化策略，依据图像重叠关系优先优化新图及其邻近图像；同时采用自适应学习率调度机制，平衡新旧图像间的训练稳定性。多组基准测试表明，本方法显著缩短训练时间，单张图像优化仅需数秒，渲染损失极低，是迈向快速渐进式3DGS重建的重要一步。
原文摘要 · Abstract (English)
3D Gaussian Splatting (3DGS) achieves high-fidelity rendering with fast real-time performance, but existing methods rely on offline training after full Structure-from-Motion (SfM) processing. In contrast, this work introduces Gaussian on-the-fly Splatting (abbreviated as On-the-Fly GS), a progressive framework enabling near real-time 3DGS optimization during image capture. As each image arrives, its pose and sparse points are updated via On-the-Fly SfM, and newly optimized Gaussians are immediately integrated into the 3DGS field. To achieve this, we propose a progressive Local & Semi-Global optimization to prioritize the new image and its neighbors by their corresponding overlapping relationship, allowing the new image and its overlapping images to get more training. To further stabilize training across previous and new images, an adaptive learning rate schedule balances the iterations and the learning rate. Extensive experiments on multiple benchmarks show that our On-the-Fly GS reduces training time significantly, optimizing each new image in seconds with minimal rendering loss, offering one of the first practical steps toward rapid, progressive 3DGS reconstruction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。