arXiv:2502.05859cs.CV2025-02被引 2

融合多种投影优势,实现高效全景深度估计

SphereFusion: Efficient Panorama Depth Estimation via Gated Fusion

  • 在球面与等距柱状投影域并行提取特征,再通过门控融合选择可靠信息
  • 在3个公开数据集上表现领先,512×1024图像推理仅需17毫秒
  • 适合需要实时全景深度估计的自动驾驶与机器人场景

随着全景相机的快速发展,全景深度估计受到计算机视觉领域的广泛关注,尤其在机器人感知和自动驾驶中具有重要应用。然而,现有方法依赖不同投影格式时面临挑战:等距柱状、立方体贴图和切线投影易产生畸变与不连续性,而球面投影则可能导致纹理细节丢失。为此,我们提出SphereFusion,一种端到端框架,融合多种投影方式的优势。具体而言,SphereFusion首先通过二维卷积与网格操作,在等距柱状投影和球面投影域中分别提取两类特征。这些特征随后被投影至球面域,由门控融合模块选择最可靠的特征进行融合。最终,全景深度在球面域内完成估计。同时,采用缓存策略提升网格操作效率。在三个公开全景数据集上的大量实验表明,SphereFusion在性能上达到当前先进水平,且推理速度最快,512×1024图像仅需17毫秒。

原文摘要 · Abstract (English)

Due to the rapid development of panorama cameras, the task of estimating panorama depth has attracted significant attention from the computer vision community, especially in applications such as robot sensing and autonomous driving. However, existing methods relying on different projection formats often encounter challenges, either struggling with distortion and discontinuity in the case of equirectangular, cubemap, and tangent projections, or experiencing a loss of texture details with the spherical projection. To tackle these concerns, we present SphereFusion, an end-to-end framework that combines the strengths of various projection methods. Specifically, SphereFusion initially employs 2D image convolution and mesh operations to extract two distinct types of features from the panorama image in both equirectangular and spherical projection domains. These features are then projected onto the spherical domain, where a gate fusion module selects the most reliable features for fusion. Finally, SphereFusion estimates panorama depth within the spherical domain. Meanwhile, SphereFusion employs a cache strategy to improve the efficiency of mesh operation. Extensive experiments on three public panorama datasets demonstrate that SphereFusion achieves competitive results with other state-of-the-art methods, while presenting the fastest inference speed at only 17 ms on a 512$\times$1024 panorama image.

全景深度门控融合实时估计

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。