用自监督方法让3D模型适应肠镜图像，提升几何估计精度
ColonAdapter: Geometry Estimation Through Foundation Model Adaptation for Colonoscopy
- 基于预训练几何模型，通过自监督微调适配肠镜场景
- 在无真实内参条件下实现顶尖的位姿与深度估计性能
- 适合医疗影像三维重建研究者，尤其关注低纹理区域建模
从单目肠镜图像中估计3D几何结构面临非朗伯表面、移动光源和大范围无纹理区域等挑战。尽管近期3D几何基础模型可避免多阶段流程，但在临床场景下性能下降。这些模型主要在自然场景数据集上训练，难以处理肠镜中的反光和均质纹理，导致几何估计不准。本文提出ColonAdapter，一种自监督微调框架，用于将几何基础模型适配至肠镜几何估计。方法利用预训练几何先验，同时针对临床数据进行定制化优化。为提升低纹理区域表现并保证尺度一致性，引入细节恢复模块（DRM）和几何一致性损失；此外，置信度加权的光度损失增强训练稳定性。在合成与真实数据集上的实验表明，该方法在相机位姿估计、单目深度预测及密集3D点云重建任务上达到当前最优性能，且无需真实内参信息。
原文摘要 · Abstract (English)
Estimating 3D geometry from monocular colonoscopy images is challenging due to non-Lambertian surfaces, moving light sources, and large textureless regions. While recent 3D geometric foundation models eliminate the need for multi-stage pipelines, their performance deteriorates in clinical scenes. These models are primarily trained on natural scene datasets and struggle with specularity and homogeneous textures typical in colonoscopy, leading to inaccurate geometry estimation. In this paper, we present ColonAdapter, a self-supervised fine-tuning framework that adapts geometric foundation models for colonoscopy geometry estimation. Our method leverages pretrained geometric priors while tailoring them to clinical data. To improve performance in low-texture regions and ensure scale consistency, we introduce a Detail Restoration Module (DRM) and a geometry consistency loss. Furthermore, a confidence-weighted photometric loss enhances training stability in clinical environments. Experiments on both synthetic and real datasets demonstrate that our approach achieves state-of-the-art performance in camera pose estimation, monocular depth prediction, and dense 3D point map reconstruction, without requiring ground-truth intrinsic parameters.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。