用3D高斯点云实现弱监督占位感知，解决渲染重复预测问题。
GSRender: Deduplicated Occupancy Prediction via Weakly Supervised 3D Gaussian Splatting
- 基于3D高斯点云简化采样，弱监督下提升效率与精度
- 新模块减少相邻帧冗余预测，RayIoU提升6.0点
- 适合自动驾驶中低成本占位估计场景
弱监督三维占位感知对户外视觉自动驾驶至关重要。以往基于NeRF的方法在样本数量上难以平衡：样本过多降低效率，过少则影响精度，导致平均交并比（mIoU）波动达5-10点。此外，即使输入多视角图像，每个视点每次仅渲染单张图像，造成预测重复，严重削弱实用性，但该问题未被充分关注。为此，我们提出GSRender，利用3D高斯点云进行弱监督占位估计，简化采样过程。同时引入射线补偿模块，通过补偿邻近帧特征以减少重复预测，并重新设计动态损失函数，消除邻近帧动态物体的影响。大量实验表明，该方法在RayIoU上达到当前最优（+6.0），且缩小了与全监督方法的差距。本工作为弱监督占位感知奠定了坚实基础。代码已公开于https://github.com/Jasper-sudo-Sun/GSRender。
原文摘要 · Abstract (English)
Weakly-supervised 3D occupancy perception is crucial for vision-based autonomous driving in outdoor environments. Previous methods based on NeRF often face a challenge in balancing the number of samples used. Too many samples can decrease efficiency, while too few can compromise accuracy, leading to variations in the mean Intersection over Union (mIoU) by 5-10 points. Furthermore, even with surrounding-view image inputs, only a single image is rendered from each viewpoint at any given moment. This limitation leads to duplicated predictions, which significantly impacts the practicality of the approach. However, this issue has largely been overlooked in existing research. To address this, we propose GSRender, which uses 3D Gaussian Splatting for weakly-supervised occupancy estimation, simplifying the sampling process. Additionally, we introduce the Ray Compensation module, which reduces duplicated predictions by compensating for features from adjacent frames. Finally, we redesign the dynamic loss to remove the influence of dynamic objects from adjacent frames. Extensive experiments show that our approach achieves SOTA results in RayIoU (+6.0), while also narrowing the gap with 3D- supervised methods. This work lays a solid foundation for weakly-supervised occupancy perception. The code is available at https://github.com/Jasper-sudo-Sun/GSRender.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。