用骨骼结构指导单目新视角合成，提升多视角一致性与精度。
Skel3D: Skeleton Guided Novel View Synthesis
- 引入骨骼引导层，结合射线条件归一化，增强生成结构合理性。
- 在Objaverse数据集上显著提升多视角一致性和姿态准确性。
- 无需显式3D表示，适合复杂物体的开放集新视角生成任务。
本文提出一种基于骨骼引导的单目开放集新视角合成方法，利用物体骨骼结构指导扩散模型生成。基于预训练2D图像生成器，该方法采用包含骨骼结构动画物体的Objaverse数据集，通过在现有射线条件归一化（RCN）层后添加骨骼引导层，提供详细结构信息，提升生成视图的质量。实验表明，该方法在Objaverse数据集上显著改善了多种物体类别下的多视角一致性与姿态准确性，优于现有最先进的新视角合成技术，且不依赖显式3D表示。
原文摘要 · Abstract (English)
In this paper, we present an approach for monocular open-set novel view synthesis (NVS) that leverages object skeletons to guide the underlying diffusion model. Building upon a baseline that utilizes a pre-trained 2D image generator, our method takes advantage of the Objaverse dataset, which includes animated objects with bone structures. By introducing a skeleton guide layer following the existing ray conditioning normalization (RCN) layer, our approach enhances pose accuracy and multi-view consistency. The skeleton guide layer provides detailed structural information for the generative model, improving the quality of synthesized views. Experimental results demonstrate that our skeleton-guided method significantly enhances consistency and accuracy across diverse object categories within the Objaverse dataset. Our method outperforms existing state-of-the-art NVS techniques both quantitatively and qualitatively, without relying on explicit 3D representations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。