arXiv:2606.22477cs.CV2026-06

让图像生成兼顾物理现实,实现多投影精准映射

Physically-guided Image Generation for Multi-Projection Mapping

论文配图:Physically-guided Image Generation for Multi-Projection Mapping
图 1 · 摘自论文原文
  • 引入协同与对抗两种生成模式,融合物理约束与创作自由
  • 在四投影实测中,几何对齐与色域利用率显著优于现有方法
  • 支持动态调整约束,适合数字孪生与交互艺术场景

投影映射(PM)可将数字内容无缝投射到真实三维物体上,是沉浸式可视化、数字孪生和互动艺术的基础技术。尽管文本到图像的扩散模型极大促进了定制内容生成,但直接集成到实际PM流程仍面临理想化二维生成与物理限制不匹配的挑战。为此,本文提出两种应用级生成范式:协同范式(使生成语义与物理属性协调)与对抗范式(通过辐射补偿消除表面干扰)。基于此,我们提出ConPhyG——一个统一可控的物理引导多投影生成框架,支持创作者交互调整物理约束并灵活切换生成模式。在协同模式下,将像素级色域、深度和边缘等多维物理先验注入扩散过程;在对抗模式下,释放生成潜力并采用有界数值优化进行多投影辐射补偿。用户可动态切换约束以平衡艺术自由与物理可行性。此外,我们通过顺序生成策略将ConPhyG扩展至360度多视角一致的投影映射。在真实四投影系统上的定量与定性评估表明,ConPhyG在几何对齐、色域利用率和语义保真度方面显著优于现有最先进方法。

原文摘要 · Abstract (English)

Projection Mapping (PM) enables seamless superimposition of digital content onto real-world 3D objects, serving as a fundamental technique for immersive visualization, digital twins, and interactive art. Although text-to-image diffusion models have greatly facilitated customized content creation, directly integrating them into practical PM pipelines remains challenging due to the mismatch between idealized 2D generation and physical constraints. To bridge this gap, this paper formalizes two application-level generative paradigms: the cooperative paradigm (harmonizing generated semantics with physical attributes) and the adversarial paradigm (eliminating surface interference via radiometric compensation). Based on this, we propose ConPhyG, a unified controllable physically-guided generative multi-projection mapping framework that enables creators to interactively adjust physical constraints and flexibly switch generative paradigms. In cooperative mode, multi-dimensional physical priors (per-pixel gamut, depth, and edges) are injected into the diffusion process. In adversarial mode, the framework releases the generative potential and applies bounded numerical optimization for multi-projector radiometric compensation. It allows users to dynamically switch constraints to balance artistic freedom with physical feasibility. Furthermore, we extend ConPhyG to 360-degree multi-view consistent PM using a sequential generation strategy. Quantitative and qualitative evaluations on a real-world four-projector setup demonstrate that ConPhyG significantly outperforms state-of-the-art methods in geometric alignment, gamut utilization, and semantic fidelity.

投影映射扩散模型物理引导多投影

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。