arXiv:2505.17338cs.CVcs.AI2025-05中稿 · ECCV被引 4

用2.8秒完成CT扫描的逼真渲染,无需逐个扫描优化。

Render-FM: Feedforward Model for Real-time Photorealistic Volumetric Rendering

  • 直接从CT体积回归6D高斯点云参数,一步完成渲染准备。
  • 单次推理仅需2.8秒,比传统方法快500倍,支持实时交互。
  • 可处理未见解剖结构和新调色函数,适合临床快速可视化。

CT扫描的逼真体渲染对临床工作流程大有裨益,但神经方法如NeRF和3DGS需耗费数小时(NeRF)或约30分钟(3DGS)进行每扫描优化,难以在临床中应用。本文提出Render-FM,一种前馈模型,可在单次2.8秒前向传播中直接从CT体积回归6D高斯点云(6DGS)参数,实现相较逐扫描优化500倍的速度提升。为弥合自然场景重建与医学体渲染之间的领域差距,引入解剖引导预热(AGP),将分割掩码和转移函数作为结构与外观先验,而现有高斯点云方法常忽略此类信息。基于nnU-Net启发的3D U-Net,在多样化CT数据上训练,预测每个体素的6DGS参数,支持即时实时渲染。相比逐扫描方法,该模型可泛化至未见解剖结构、新转移函数,并实现组合器官可视化,无需额外准备时间。可选89秒微调进一步提升质量,超越逐扫描优化基线。项目页:https://gaozhongpai.github.io/renderfm/。

原文摘要 · Abstract (English)

Photorealistic volumetric rendering of CT scans greatly benefits clinical workflows, yet neural approaches such as Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) require prohibitive per-scan optimization (hours for NeRF, about 30 minutes for 3DGS), making them impractical in clinical settings. We propose Render-FM, a feedforward model that eliminates this bottleneck by directly regressing 6D Gaussian Splatting (6DGS) parameters from a CT volume in a single 2.8-second forward pass, a 500x speedup over per-scan optimization. To bridge the domain gap between natural scene reconstruction and medical volumetric rendering, we introduce Anatomy-Guided Priming (AGP), which incorporates segmentation masks and transfer functions as structural and appearance priors, information that existing Gaussian splatting methods overlook. Built on an nnU-Net-inspired 3D U-Net trained on diverse CT scans, Render-FM predicts per-voxel 6DGS parameters and supports immediate real-time rendering. Unlike per-scan methods, it generalizes to unseen anatomies, novel transfer functions, and enables compositional organ visualization with zero additional preparation time. Optional 89-second fine-tuning further improves quality, surpassing per-scan optimized baselines. Project page: https://gaozhongpai.github.io/renderfm/.

医学图像实时渲染高斯点云3D生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。