用单目内窥镜视频重建动态组织的3D模型，提升手术导航精度。
NeRFscopy: Neural Radiance Fields for in-vivo Time-Varying Tissues from Endoscopy
- 基于可变形神经辐射场，结合时间变化的运动参数建模。
- 在多个挑战性场景中实现更优的新视角合成效果。
- 无需模板或预训练模型，适合临床实时重建需求。
内窥镜在医学影像中至关重要，用于诊断、预后和治疗。开发鲁棒的动态3D重建方法对内窥镜视频可提升可视化效果，改善诊断准确性，辅助治疗规划并指导手术。然而，由于组织形变、单目相机、光照变化、遮挡及未知相机轨迹，该任务面临挑战。受神经渲染启发，本文提出NeRFscopy，一种自监督的新型视图合成与3D重建管道，用于从单目视频重建可变形的内窥镜组织。该方法包含一个规范化的辐射场与随时间变化的形变场，通过SE(3)变换参数化。同时，通过引入复杂项高效利用颜色图像，仅从数据学习3D隐式模型，无需任何模板或预训练模型。NeRFscopy在多种具有挑战性的内窥镜场景中实现了精确的新视角合成，优于现有方法。
原文摘要 · Abstract (English)
Endoscopy is essential in medical imaging, used for diagnosis, prognosis and treatment. Developing a robust dynamic 3D reconstruction pipeline for endoscopic videos could enhance visualization, improve diagnostic accuracy, aid in treatment planning, and guide surgery procedures. However, challenges arise due to the deformable nature of the tissues, the use of monocular cameras, illumination changes, occlusions and unknown camera trajectories. Inspired by neural rendering, we introduce NeRFscopy, a self-supervised pipeline for novel view synthesis and 3D reconstruction of deformable endoscopic tissues from a monocular video. NeRFscopy includes a deformable model with a canonical radiance field and a time-dependent deformation field parameterized by SE(3) transformations. In addition, the color images are efficiently exploited by introducing sophisticated terms to learn a 3D implicit model without assuming any template or pre-trained model, solely from data. NeRFscopy achieves accurate results in terms of novel view synthesis, outperforming competing methods across various challenging endoscopy scenes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。