用轨迹对齐解决多摄像头异步动态3D重建的模糊与失真问题。
ASTRA: Asynchronous Spatio-Temporal Reconstruction via Trajectory Alignment
- 通过2D运动轨迹作为显式监督,避免纹理缺失导致的对齐失败。
- 在25帧异步下提升1.4 dB PSNR,时序误差降低54%,同步成功率接近翻倍。
- 适合真实场景中摄像头不同步的动态3D重建任务。
动态3D场景重建在严格同步多相机输入下已取得显著进展。然而,现实场景中摄像设备间存在时间异步,导致严重运动模糊和几何伪影。现有异步重建方法通常依赖光度监督估计时序偏移,但在大偏移和复杂运动下,外观匹配提供的时序线索较弱。我们指出两大瓶颈:纹理缺失导致的对齐信号消失,以及形变引起的时序误差被吸收进扭曲几何或运动中。为此,提出ASTRA(基于轨迹对齐的异步时空重建),引入2D运动轨迹作为显式、与纹理无关的监督信号,联合优化时序偏移与动态3D表示,通过投影3D点运动与观测2D轨迹对齐实现同步。结合动态掩码与置信度掩码抑制不可靠轨迹约束。在多种动态高斯溅射骨干网络上实验表明,ASTRA在高达25帧异步条件下仍能保持高频空间细节,实现约1.4 dB PSNR提升,时序偏移平均绝对误差降低54.0%,同步成功率达近四倍。
原文摘要 · Abstract (English)
Dynamic 3D scene reconstruction has achieved remarkable success under the assumption of strictly synchronized multi-camera inputs. However, in real-world scenarios, temporal asynchrony among capturing devices remains a critical challenge, leading to severe motion blur and geometric artifacts. Existing asynchronous reconstruction methods typically estimate temporal offsets through photometric supervision, but appearance matching provides weak temporal cues under large offsets and complex motions. We attribute this limitation to two major bottlenecks: texture-induced collapse, where low-texture regions provide nearly vanishing alignment signals, and deformation-induced coupling, where temporal errors are absorbed into distorted geometry or motion rather than being explicitly corrected. To address these issues, we propose ASTRA (Asynchronous Spatio-Temporal Reconstruction via Trajectory Alignment), a framework that introduces 2D motion trajectories as explicit, texture-agnostic supervision for asynchronous dynamic reconstruction. Instead of synchronizing cameras solely through rendered color residuals, ASTRA jointly optimizes temporal offsets and dynamic 3D representations by aligning the projected motion of reconstructed 3D points with observed 2D trajectories, while using dynamic and certainty masking to suppress unreliable trajectory constraints. Extensive experiments on different dynamic Gaussian Splatting backbones show that ASTRA preserves high-frequency spatial details and sustains strong robustness even under severe asynchrony with up to 25-frame offsets, achieving approximately 1.4 dB PSNR improvement, reducing temporal-offset MAE by 54.0\%, and nearly quadrupling the synchronization success rate.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。