仅用一张术中图像+术前MRI,快速重建高保真手术场景3D模型。
Surgical Neural Radiance Fields from One Image
- 用术前MRI生成视角集,通过风格迁移融合术中图像
- 四例神经外科案例验证,重建质量接近真实显微图像
- 适合术中实时3D重建,无需多视角采集
神经辐射场(NeRF)在3D重建与视图合成方面表现优异,但其依赖大量多视角数据,在术中环境下难以应用,因时间限制无法获取足够数据。本文提出一种新方法,仅需一张术中图像和术前MRI即可高效训练适用于手术场景的NeRF。利用术前MRI确定相机视角和训练图像集,术中通过神经风格迁移(结合WTC2与STROTSS)将术中图像外观迁移到预构建数据集,避免过度风格化,从而实现快速单图像NeRF训练。在四个临床神经外科案例中评估,定量对比真实显微图像训练的NeRF模型,结果显示合成结果具有高相似性,结构相似性指标表明重建精度良好且纹理保留完整。该方法证明了在手术场景下实现单图像NeRF训练的可行性,突破了传统多视角方法的局限。
原文摘要 · Abstract (English)
Purpose: Neural Radiance Fields (NeRF) offer exceptional capabilities for 3D reconstruction and view synthesis, yet their reliance on extensive multi-view data limits their application in surgical intraoperative settings where only limited data is available. In particular, collecting such extensive data intraoperatively is impractical due to time constraints. This work addresses this challenge by leveraging a single intraoperative image and preoperative data to train NeRF efficiently for surgical scenarios. Methods: We leverage preoperative MRI data to define the set of camera viewpoints and images needed for robust and unobstructed training. Intraoperatively, the appearance of the surgical image is transferred to the pre-constructed training set through neural style transfer, specifically combining WTC2 and STROTSS to prevent over-stylization. This process enables the creation of a dataset for instant and fast single-image NeRF training. Results: The method is evaluated with four clinical neurosurgical cases. Quantitative comparisons to NeRF models trained on real surgical microscope images demonstrate strong synthesis agreement, with similarity metrics indicating high reconstruction fidelity and stylistic alignment. When compared with ground truth, our method demonstrates high structural similarity, confirming good reconstruction quality and texture preservation. Conclusion: Our approach demonstrates the feasibility of single-image NeRF training in surgical settings, overcoming the limitations of traditional multi-view methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。