多智能体协同实时重建3D场景,精度和速度双突破。
MAC-Ego3D: Multi-Agent Gaussian Consensus for Real-Time Collaborative Ego-Motion and Photorealistic 3D Reconstruction
- 用高斯点云统一表示局部地图,支持高效渲染与对齐。
- 实现15倍推理加速,定位误差降低一个数量级,图像质量提升4-10dB。
- 适合需要实时高保真3D重建的自动驾驶、机器人等场景。
实时多智能体协同进行自我运动估计与高保真3D重建对可扩展空间智能至关重要。传统方法生成稀疏低细节地图,而现有密集映射方法存在高延迟问题。为此,我们提出MAC-Ego3D框架,通过多智能体高斯共识实现实时协同光栅化3D重建。各智能体独立构建、对齐并迭代优化本地地图,采用统一的高斯点表示。通过智能体内高斯共识,在单个智能体内强制邻近高斯点间的空间一致性;通过并行化的跨智能体高斯共识,异步对齐与优化本地地图,正则化多智能体高斯点,无缝融合为高保真3D模型。利用高斯原语,MAC-Ego3D支持高效的RGB-D渲染,实现快速跨智能体高斯关联与对齐。该方法兼顾局部精度与全局一致性,显著提升效率,大幅降低定位误差,增强建模保真度。在合成与真实世界基准上均达到新SOTA,推理速度提升15倍,部分情况下的自运动估计误差降低一个数量级,RGB PSNR提升4至10 dB。代码将公开于https://github.com/Xiaohao-Xu/MAC-Ego3D。
原文摘要 · Abstract (English)
Real-time multi-agent collaboration for ego-motion estimation and high-fidelity 3D reconstruction is vital for scalable spatial intelligence. However, traditional methods produce sparse, low-detail maps, while recent dense mapping approaches struggle with high latency. To overcome these challenges, we present MAC-Ego3D, a novel framework for real-time collaborative photorealistic 3D reconstruction via Multi-Agent Gaussian Consensus. MAC-Ego3D enables agents to independently construct, align, and iteratively refine local maps using a unified Gaussian splat representation. Through Intra-Agent Gaussian Consensus, it enforces spatial coherence among neighboring Gaussian splats within an agent. For global alignment, parallelized Inter-Agent Gaussian Consensus, which asynchronously aligns and optimizes local maps by regularizing multi-agent Gaussian splats, seamlessly integrates them into a high-fidelity 3D model. Leveraging Gaussian primitives, MAC-Ego3D supports efficient RGB-D rendering, enabling rapid inter-agent Gaussian association and alignment. MAC-Ego3D bridges local precision and global coherence, delivering higher efficiency, largely reducing localization error, and improving mapping fidelity. It establishes a new SOTA on synthetic and real-world benchmarks, achieving a 15x increase in inference speed, order-of-magnitude reductions in ego-motion estimation error for partial cases, and RGB PSNR gains of 4 to 10 dB. Our code will be made publicly available at https://github.com/Xiaohao-Xu/MAC-Ego3D .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。