arXiv:2601.11508cs.CV2026-01被引 2

让3D场景中物体随时间变化时保持身份一致,提升动态室内场景理解能力。

ReScene4D: Temporally Consistent Semantic Instance Segmentation of Evolving Indoor 3D Scenes

  • 利用时空对比损失与掩码机制共享时间信息,实现稀疏观测下的连续跟踪
  • 在3RScan数据集上达到当前最优,提升实例分割与时间一致性表现
  • 适合研究动态环境建模、机器人导航与智能空间感知的开发者

室内环境随物体移动、出现或消失而持续演变。捕捉此类动态需在间断获取的3D扫描中维持实例身份的一致性,即使变化未被直接观测。我们提出并形式化了稀疏时间维度4D室内语义实例分割(SIS)任务,联合完成分割、识别与时间关联。现有3DSIS方法因缺乏时间推理,依赖离散匹配步骤;而4D LiDAR方法则因依赖高频时间测量,在长期室内演化中表现不佳。我们提出ReScene4D,一种无需密集观测即可适配3DSIS架构的4DSIS新方法。通过时空对比损失、掩码与序列化策略,自适应利用几何与语义先验进行时间信息共享,实现一致实例追踪并提升标准3DSIS性能。为评估该任务,我们定义新指标t-mAP,扩展mAP以奖励时间身份一致性。ReScene4D在3RScan数据集上取得领先结果,建立理解动态室内场景的新基准。

原文摘要 · Abstract (English)

Indoor environments evolve as objects move, appear, or leave the scene. Capturing these dynamics requires maintaining temporally consistent instance identities across intermittently captured 3D scans, even when changes are unobserved. We introduce and formalize the task of temporally sparse 4D indoor semantic instance segmentation (SIS), which jointly segments, identifies, and temporally associates object instances. This setting poses a challenge for existing 3DSIS methods, which require a discrete matching step due to their lack of temporal reasoning, and for 4D LiDAR approaches, which perform poorly due to their reliance on high-frequency temporal measurements that are uncommon in the longer-horizon evolution of indoor environments. We propose ReScene4D, a novel method that adapts 3DSIS architectures for 4DSIS without needing dense observations. Our method enables temporal information sharing--using spatiotemporal contrastive loss, masking, and serialization--to adaptively leverage geometric and semantic priors across observations. This shared context enables consistent instance tracking and improves standard 3DSIS performance. To evaluate this task, we define a new metric, t-mAP, that extends mAP to reward temporal identity consistency. ReScene4D achieves state-of-the-art performance on the 3RScan dataset, establishing a new benchmark for understanding evolving indoor scenes.

4D分割实例追踪动态场景3D理解

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。