用随机采样和查询优化,让动态神经体积数据实时渲染
A Query-Efficient Stochastic Volume Rendering Framework for Time-Varying Implicit Neural Volumes

- 通过增量追踪与查询剪枝,大幅减少神经网络调用次数
- 在RTX 4090上实现1024x1024分辨率下30-40帧/秒的渲染速度
- 支持毫秒级时间步更新,可交互探索连续动态数据
动态隐式神经表示(INRs)能以紧凑形式表达科学体数据,在动态X射线计算机断层扫描等模态中往往是唯一可行方案。但传统体渲染方法因依赖昂贵的神经推理而难以实现交互式渲染。现有缓解手段如重采样、缓存或再训练会牺牲便利性与精度,对时变数据不实用。本文提出一种基于增量追踪的查询高效随机体渲染框架,采用四阶段流水线,利用光线追踪核心进行遍历、张量核心批量执行神经计算。通过光线预算分配与查询剪枝策略,显著降低INR查询次数,提升每帧性能。系统可直接从原始表示渲染多个时变INR,实现在RTX 4090上1024x1024分辨率下约30-40 FPS的流畅渲染,并收敛至高保真图像。同时支持连续时空域的交互式探索,时间步更新仅需约1-2毫秒。
原文摘要 · Abstract (English)
Time-varying implicit neural representations (INRs) provide a compact representation of scientific volumes and, for modalities such as dynamic X-ray computed tomography (CT), are often the only practical way to represent the data. However, interactive volume rendering of INRs is challenging, as cheap memory lookups are replaced by expensive neural inferences, hindering the performance. Therefore, conventional volume rendering methods such as ray marching with dense sampling are often impractical. While resampling, caching, and retraining can mitigate this cost, they compromise convenience and accuracy and become impractical for time-varying data. We tackle these challenges using a query-efficient stochastic volume rendering framework based on delta tracking. Our system employs a four-stage pipeline that exploits heterogeneous parallelism, using ray tracing cores for traversal and tensor cores for batched neural evaluation. Furthermore, we present strategies to reduce INR queries via ray budgeting and query pruning, thereby increasing per-frame performance. Using our renderer, many time-varying INRs can be rendered directly from their original representation. The system achieves ~30-40 FPS at 1024x1024 resolution on an RTX 4090 GPU and converges to high-fidelity images. Moreover, the system enables interactive temporal exploration of the continuous domain, with timestep updates taking approximately 1-2 ms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。