用光照结构图生成真实内镜图像，提升深度估计效果
PPS-Ctrl: Controllable Sim-to-Real Translation for Colonoscopy Depth Estimation
- 结合稳定扩散与控制网，用像素级光照图约束生成
- 生成图像更真实，深度估计误差比基线低12.3%
- 适合做医学影像仿真和真实场景泛化研究
精准的深度估计能提升内窥镜导航与诊断能力，但临床中获取真实深度标签困难。常采用合成数据训练，但域差距限制了在真实数据上的泛化性能。本文提出一种新型图像到图像转换框架，在保持结构的同时从临床数据生成逼真纹理。核心创新在于将稳定扩散模型与ControlNet结合，以像素级光照（PPS）图的潜在表示为条件。PPS捕捉表面光照效应，相比深度图提供更强的结构约束。实验表明，该方法生成的图像更真实，深度估计性能优于基于GAN的MI-CycleGAN。代码已开源：https://github.com/anaxqx/PPS-Ctrl。
原文摘要 · Abstract (English)
Accurate depth estimation enhances endoscopy navigation and diagnostics, but obtaining ground-truth depth in clinical settings is challenging. Synthetic datasets are often used for training, yet the domain gap limits generalization to real data. We propose a novel image-to-image translation framework that preserves structure while generating realistic textures from clinical data. Our key innovation integrates Stable Diffusion with ControlNet, conditioned on a latent representation extracted from a Per-Pixel Shading (PPS) map. PPS captures surface lighting effects, providing a stronger structural constraint than depth maps. Experiments show our approach produces more realistic translations and improves depth estimation over GAN-based MI-CycleGAN. Our code is publicly accessible at https://github.com/anaxqx/PPS-Ctrl.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。