递归融合多状态3D场景,实现可交互的动态环境建模
RecurGS: Interactive Scene Modeling via Discrete-State Recurrent Gaussian Fusion
- 基于离散状态递归融合,逐步整合多时序3D高斯表示
- 支持物体级操作与新状态合成,更新效率提升显著
- 适合构建持续演化的智能机器人感知环境
近期3D场景表示技术已实现高保真新视角合成,但应对离散场景变化及构建交互式3D环境仍是视觉与机器人领域的开放挑战。现有方法仅支持单一场景更新,无法合成新状态;部分基于扩散模型的方法虽能解耦物体与背景,但仅限单状态处理,难以跨观测融合信息。为此,我们提出RecurGS,一种递归融合框架,可将离散高斯场景状态逐次整合为持续演进的统一表示。RecurGS检测连续状态间的物体级变化,利用语义对应与基于李代数的SE(3)精修对齐几何运动,并通过回放监督实现历史结构保留的递归更新。一个体素化、可见性感知的融合模块选择性引入新观测区域,固定稳定区域,缓解灾难性遗忘,实现高效长时程更新。RecurGS支持物体级操作,无需额外扫描即可合成新场景状态,在合成与真实数据集上均保持逼真的视觉保真度。大量实验表明,该框架在重建质量与更新效率方面均有显著提升,为持续交互的高斯世界提供了可扩展解决方案。
原文摘要 · Abstract (English)
Recent advances in 3D scene representations have enabled high-fidelity novel view synthesis, yet adapting to discrete scene changes and constructing interactive 3D environments remain open challenges in vision and robotics. Existing approaches focus solely on updating a single scene without supporting novel-state synthesis. Others rely on diffusion-based object-background decoupling that works on one state at a time and cannot fuse information across multiple observations. To address these limitations, we introduce RecurGS, a recurrent fusion framework that incrementally integrates discrete Gaussian scene states into a single evolving representation capable of interaction. RecurGS detects object-level changes across consecutive states, aligns their geometric motion using semantic correspondence and Lie-algebra based SE(3) refinement, and performs recurrent updates that preserve historical structures through replay supervision. A voxelized, visibility-aware fusion module selectively incorporates newly observed regions while keeping stable areas fixed, mitigating catastrophic forgetting and enabling efficient long-horizon updates. RecurGS supports object-level manipulation, synthesizes novel scene states without requiring additional scans, and maintains photorealistic fidelity across evolving environments. Extensive experiments across synthetic and real-world datasets demonstrate that our framework delivers high-quality reconstructions with substantially improved update efficiency, providing a scalable step toward continuously interactive Gaussian worlds.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。