利用内镜光照特性提升医学3D重建质量，实现更真实、可调控的视觉效果。
MedGS: Gaussian Splatting for Multi-Modal 3D Medical Imaging
- 分离光照与组织属性，用专用MLP建模复杂光效
- 在公开与自建数据集上均优于基线方法，减少视图伪影
- 支持组织修改且保持物理光照响应，适合临床仿真应用
内镜检查是诊断消化道、泌尿生殖系统及呼吸道严重疾病的关键手段。从内镜图像中进行3D重建和新视角合成，有助于提升诊断能力。结合生理形变与内镜交互，可基于真实视频数据构建模拟工具。然而，受限的相机轨迹和视图依赖性光照导致重建失真或过拟合。本文提出MedGS，利用内镜成像中光源与相机高度对齐的独特特性，将光照效应与组织属性解耦。通过引入物理驱动的可重光照模型增强3D高斯点阵，使用专用MLP捕捉复杂光照现象，显著降低伪影并提升新视角泛化能力。在公开与自建数据集上，MedGS均取得优于基线的重建质量。相较于现有方法，其可在保留物理光照真实性的同时实现组织修改，更贴近真实临床应用场景。
原文摘要 · Abstract (English)
Endoluminal endoscopic procedures are essential for diagnosing colorectal cancer and other severe conditions in the digestive tract, urogenital system, and airways. 3D reconstruction and novel-view synthesis from endoscopic images are promising tools for enhancing diagnosis. Moreover, integrating physiological deformations and interaction with the endoscope enables the development of simulation tools from real video data. However, constrained camera trajectories and view-dependent lighting create artifacts, leading to inaccurate or overfitted reconstructions. We present MedGS, a novel 3D reconstruction framework leveraging the unique property of endoscopic imaging, where a single light source is closely aligned with the camera. Our method separates light effects from tissue properties. MedGS enhances 3D Gaussian Splatting with a physically based relightable model. We boost the traditional light transport formulation with a specialized MLP capturing complex light-related effects while ensuring reduced artifacts and better generalization across novel views. MedGS achieves superior reconstruction quality compared to baseline methods on both public and in-house datasets. Unlike existing approaches, MedGS enables tissue modifications while preserving a physically accurate response to light, making it closer to real-world clinical use. Repository: https://github.com/gmum/MedGS
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。