通过图像转黏土技术,让反射物体显出真实几何形状。
Pygmalion Effect in Vision: Image-to-Clay Translation for Reflective Geometry Reconstruction
- 用双分支网络分离反射与几何,黏土分支稳定表面结构
- 在合成黏土图像监督下,法线精度和网格完整性显著提升
- 适合做反射物体三维重建的研究者或工业应用
理解反射一直是3D重建中的长期挑战,因为外观与几何在视点依赖的反射下纠缠不清。本文提出视觉中的「皮格马利翁效应」,一种新颖框架,通过图像到黏土的转换,将反射物体“雕塑”为类黏土形态。受皮格马利翁神话启发,该方法学习抑制镜面特征,同时保持内在几何一致性,从而从含复杂反射的多视角图像中实现鲁棒重建。具体地,设计了一个双分支网络:基于BRDF的反射分支与受黏土引导的分支协同工作,后者稳定几何并优化表面法线。两个分支联合训练于合成的黏土样图像,提供无反射的中性监督信号,补充原始反射视图。在合成与真实数据集上的实验表明,本方法在法线精度与网格完整性上显著优于现有反射处理方法。此外,研究揭示:将辐射转化为中性状态,可作为反射物体几何学习的强大归纳偏置。
原文摘要 · Abstract (English)
Understanding reflection remains a long-standing challenge in 3D reconstruction due to the entanglement of appearance and geometry under view-dependent reflections. In this work, we present the Pygmalion Effect in Vision, a novel framework that metaphorically "sculpts" reflective objects into clay-like forms through image-to-clay translation. Inspired by the myth of Pygmalion, our method learns to suppress specular cues while preserving intrinsic geometric consistency, enabling robust reconstruction from multi-view images containing complex reflections. Specifically, we introduce a dual-branch network in which a BRDF-based reflective branch is complemented by a clay-guided branch that stabilizes geometry and refines surface normals. The two branches are trained jointly using the synthesized clay-like images, which provide a neutral, reflection-free supervision signal that complements the reflective views. Experiments on both synthetic and real datasets demonstrate substantial improvement in normal accuracy and mesh completeness over existing reflection-handling methods. Beyond technical gains, our framework reveals that seeing by unshining, translating radiance into neutrality, can serve as a powerful inductive bias for reflective object geometry learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。