用平面扫描提升室内三维检测精度，解决深度估计不准问题。
MVSDet: Multi-View Indoor 3D Object Detection via Efficient Plane Sweeps
- 用概率采样与软权重机制优化平面扫掠的特征放置
- 在ScanNet和ARKitScenes上实现优于NeRF的方法
- 适合关注多视角3D检测与深度估计的开发者
多视角室内三维目标检测的关键挑战在于从图像中推断精确的几何信息以实现精准检测。以往方法依赖NeRF进行几何推理，但其提取的几何信息通常不准确，导致检测性能受限。本文提出MVSDet，采用平面扫掠实现几何感知的3D目标检测。为避免大量深度平面带来的计算负担，设计了概率采样与软权重机制，根据概率体积选择每个像素的多个高分位置，并以概率得分表示置信度。进一步引入近期的像素对齐高斯点阵技术，对深度预测进行正则化，在几乎无额外计算开销下提升检测性能。在ScanNet和ARKitScenes数据集上的大量实验表明，本模型具有显著优势。代码已公开于https://github.com/Pixie8888/MVSDet。
原文摘要 · Abstract (English)
The key challenge of multi-view indoor 3D object detection is to infer accurate geometry information from images for precise 3D detection. Previous method relies on NeRF for geometry reasoning. However, the geometry extracted from NeRF is generally inaccurate, which leads to sub-optimal detection performance. In this paper, we propose MVSDet which utilizes plane sweep for geometry-aware 3D object detection. To circumvent the requirement for a large number of depth planes for accurate depth prediction, we design a probabilistic sampling and soft weighting mechanism to decide the placement of pixel features on the 3D volume. We select multiple locations that score top in the probability volume for each pixel and use their probability score to indicate the confidence. We further apply recent pixel-aligned Gaussian Splatting to regularize depth prediction and improve detection performance with little computation overhead. Extensive experiments on ScanNet and ARKitScenes datasets are conducted to show the superiority of our model. Our code is available at https://github.com/Pixie8888/MVSDet.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。