用圆柱坐标解决360度图像生成的对称伪影问题
CylinderPlane: Nested Cylinder Representation for 3D-aware Image Generation
- 基于圆柱坐标系设计新表征,分离不同角度特征
- 支持无伪影的360度高质量图像生成,多尺度嵌套结构增强细节捕捉
- 兼容任意神经渲染管线,适合复杂场景生成任务
尽管三平面表示推动了3D感知图像生成模型的发展,但其固有结构导致对称区域共享特征,引发多面体伪影,限制了360°视角图像生成。本文提出CylinderPlane,一种基于圆柱坐标系的新型隐式表征,消除特征歧义,保障360°多视角一致性。与笛卡尔坐标系下三平面表示必然存在的特征纠缠不同,圆柱坐标系显式分离不同角度特征,使生成结果具备高质量、无伪影的360°图像合成能力。进一步引入多尺度嵌套圆柱结构,提升模型对复杂几何和不同分辨率的适应性。不同分辨率的圆柱组合可有效捕捉关键位置与多尺度特征,显著促进细节学习并增强对分辨率变化的鲁棒性。该表征与隐式渲染方法无关,可轻松集成至任意神经渲染流程。在合成数据集和真实野外图像上的大量实验表明,所提方法优于现有方法。
原文摘要 · Abstract (English)
While the proposal of the Tri-plane representation has advanced the development of the 3D-aware image generative models, problems rooted in its inherent structure, such as multi-face artifacts caused by sharing the same features in symmetric regions, limit its ability to generate 360$^\circ$ view images. In this paper, we propose CylinderPlane, a novel implicit representation based on Cylindrical Coordinate System, to eliminate the feature ambiguity issue and ensure multi-view consistency in 360$^\circ$. Different from the inevitable feature entanglement in Cartesian coordinate-based Tri-plane representation, the cylindrical coordinate system explicitly separates features at different angles, allowing our cylindrical representation possible to achieve high-quality, artifacts-free 360$^\circ$ image synthesis. We further introduce the nested cylinder representation that composites multiple cylinders at different scales, thereby enabling the model more adaptable to complex geometry and varying resolutions. The combination of cylinders with different resolutions can effectively capture more critical locations and multi-scale features, greatly facilitates fine detail learning and robustness to different resolutions. Moreover, our representation is agnostic to implicit rendering methods and can be easily integrated into any neural rendering pipeline. Extensive experiments on both synthetic dataset and unstructured in-the-wild images demonstrate that our proposed representation achieves superior performance over previous methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。