通过分离镜面反射，实现结肠镜图像的自监督深度估计。
SHADeS: Self-supervised Monocular Depth Estimation Through Non-Lambertian Image Decomposition
- 将镜面反射作为独立光照成分处理，改进传统光照模型。
- 在真实结肠镜数据上，深度与光照分解均优于现有方法。
- 适合需要精准三维重建的内窥镜导航与病灶分析场景。
视觉3D场景重建可支持结肠镜导航，帮助识别已检查区域并刻画息肉的大小与形状。由于光照变化复杂，尤其是大量镜面反射，该问题仍极具挑战性。本文研究如何有效解耦光照与深度。提出一种自监督模型（SHADeS），从单张图像中同时估计阴影、反照率、深度和镜面反射（SHADeS）。不同于以往方法（IID），采用非朗伯模型，将镜面反射视为独立光成分。在真实结肠镜数据集（Hyper Kvasir）上，先前的光照分解（IID）与深度估计（MonoVIT, ModoDepth2）受镜面反射干扰严重；而SHADeS能生成对镜面区域鲁棒的光照分解与深度图。在模拟数据集（C3VD）上进一步验证了模型的鲁棒性。结果表明，建模镜面反射可显著提升结肠镜深度估计性能。提出的自监督方法有效结合光照解耦与深度估计，光照分解亦有望用于结肠内定位识别等任务。
原文摘要 · Abstract (English)
Purpose: Visual 3D scene reconstruction can support colonoscopy navigation. It can help in recognising which portions of the colon have been visualised and characterising the size and shape of polyps. This is still a very challenging problem due to complex illumination variations, including abundant specular reflections. We investigate how to effectively decouple light and depth in this problem. Methods: We introduce a self-supervised model that simultaneously characterises the shape and lighting of the visualised colonoscopy scene. Our model estimates shading, albedo, depth, and specularities (SHADeS) from single images. Unlike previous approaches (IID), we use a non-Lambertian model that treats specular reflections as a separate light component. The implementation of our method is available at https://github.com/RemaDaher/SHADeS. Results: We demonstrate on real colonoscopy images (Hyper Kvasir) that previous models for light decomposition (IID) and depth estimation (MonoVIT, ModoDepth2) are negatively affected by specularities. In contrast, SHADeS can simultaneously produce light decomposition and depth maps that are robust to specular regions. We also perform a quantitative comparison on phantom data (C3VD) where we further demonstrate the robustness of our model. Conclusion: Modelling specular reflections improves depth estimation in colonoscopy. We propose an effective self-supervised approach that uses this insight to jointly estimate light decomposition and depth. Light decomposition has the potential to help with other problems, such as place recognition within the colon.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。