arXiv:2607.17803cs.CV2026-07

用前馈方式实现投影-相机系统的高保真逆渲染,速度提升千倍以上。

FF-ProCams: Feed-Forward Gaussian Splatting for Projector-Camera System

论文配图:FF-ProCams: Feed-Forward Gaussian Splatting for Projector-Camera System
图 1 · 摘自论文原文
  • 采用混合Mamba2-Transformer架构,融合多视角几何与光照信息
  • 仅需8张输入图,0.13秒完成重建,精度超优化类方法(297视图)
  • 支持新投影图案和视角,适合空间增强现实等实时应用

投影-相机(ProCams)系统通过结构化照明实现主动场景感知与可控外观操控,是空间增强现实、投影映射和表面反射率获取的核心基础设施。现有逆渲染方法虽精度高,但依赖耗时的每场景优化;主流前馈3D重建模型则生成固化外观,无法适应空间变化的投影光照。为解决精度与效率的权衡,本文提出FF-ProCams,一种面向ProCams的前馈3D高斯逆渲染框架。采用混合Mamba2-Transformer编码器,从稀疏多视角观测中聚合跨视图几何与光度线索,轻量级解码头在单次前向传播中预测可重光照的高斯表示。进一步设计投影器感知的可微渲染器,可在任意主动照明和ProCams位姿下合成相机观测。为支持前馈训练,构建大规模合成ProCams数据集,覆盖多样物体几何与表面材质。实验表明,FF-ProCams实现高保真投影感知渲染,泛化至未见图案,支持新投影-相机位姿。仅用8个输入视图,其性能超越基于优化的基线(297视图),测试重建时间降至0.13秒,提速三至五个数量级。代码与数据已公开于https://github.com/CPREgroup/FF-ProCams/。

原文摘要 · Abstract (English)

Projector-camera (ProCams) systems achieve active scene perception and controllable appearance manipulation via structured illumination, serving as a core infrastructure for spatial augmented reality, projection mapping, and surface reflectance acquisition. Existing inverse-rendering methods for ProCams deliver high-fidelity results but rely on time-consuming per-scene optimization, while mainstream feed-forward 3D reconstruction models produce baked appearance that cannot adapt to spatially varying projector illumination. To resolve this accuracy-efficiency trade-off, we propose FF-ProCams, a Feed-Forward 3D Gaussian inverse-rendering framework for ProCams. A hybrid Mamba2-Transformer encoder aggregates cross-view geometric and photometric cues from sparse multi-view observations, and lightweight heads predict a relightable Gaussian representation in a single forward pass. We further design a projector-aware differentiable renderer to synthesize camera observations under arbitrary active illumination and ProCams poses. To enable feed-forward training, we construct a large-scale synthetic ProCams dataset covering diverse object geometries and surface materials. Experiments show FF-ProCams achieves high-fidelity projector-aware rendering, generalizes to unseen patterns, and supports novel projector-camera poses. Using only 8 input views, it outperforms optimization-based baselines with 297 views while reducing test-time reconstruction to 0.13 seconds (a three-to-five-order-of-magnitude speedup). The code and data are available at https://github.com/CPREgroup/FF-ProCams/.

3D重建投影映射前馈渲染高斯溅射

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。