优化了非视域成像的GPU流水线,大幅提速并降低内存占用。
Memory-efficient GPU pipelines for real-time non-line-of-sight reconstruction

- 重构波基算法流水线,融合核函数、分块变换与低精度存储。
- 比基准快42倍,内存使用降至2.5%,支持更精细重建。
- 为下一代非视域视频处理提供帧率预算和去噪新策略。
非视域(NLOS)成像通过单光子雪崩二极管(SPAD)记录的间接光,重建被遮挡场景。单次重建是一个大规模逆问题:需对数十亿个光子时间戳进行分箱、内存传输、变换与求逆。随着SPAD阵列提升采集吞吐量,重建成为瓶颈。本文重新设计两种经典波基算法——f-k迁移与相位场(phasor-fields)的GPU执行流程,支持流式与离线处理。在相位场方面,预先离线计算环形与径向核函数,利用环的解析傅里叶变换,避免运行时存储密集核矩阵,显著降低内存与带宽开销。通过融合核函数、基于线程束的光子分箱、批量变换、CUDA图重放及仅在关键路径使用FP16存储,实现高效流水线。所提实现比参考流式管道快42倍,比最快已发表基线快14倍,同时内存占用下降至2.5%。该优化使相同硬件上可进行更大更精细的重建,或在极低内存预算下达成类似效果。报告各优化选择的消融实验,并提出三种由帧率提升带来的去噪策略,适用于下一代NLOS视频处理。
原文摘要 · Abstract (English)
Non-line-of-sight (NLOS) imaging reconstructs scenes hidden around a corner from indirect light recorded by a single-photon avalanche diode (SPAD). A single reconstruction is a large inverse problem: billions of photon timestamps must be binned, moved through memory, transformed and inverted. As SPAD arrays raise acquisition throughput, reconstruction becomes the limiting stage. We rebuild the GPU execution of two established wave-based algorithms, f-k migration and phasor-fields, for both streaming and offline processing. On the phasor-fields side we assemble the ring-and-radius kernels of previous work once and offline, using the analytic Fourier transform of a ring, so the propagation kernel never exists in dense form at runtime, reducing the memory and bandwidth. We reorganize the pipeline of both algorithms with fused kernels, warp-level photon binning, batched transforms, CUDA graph replay, and FP16 storage applied only where it reduces the actual bottleneck. Our implementations are up to 42x faster than the reference streaming pipeline and up to 14x faster than the fastest published GPU baseline, all while using a fraction of the memory (down to 2.5%), enabling vastly larger and finer reconstructions on the same hardware, or comparable ones within a much lower memory budget. We report an ablation of each implementation choice and propose three denoising strategies enabled by the resulting frame budget for next-generation NLOS video processing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。