从一张图生成360度人物旋转视频并重建高保真3D模型
HumanOrbit: 3D Human Reconstruction as 360° Orbit Generation
- 基于视频扩散模型生成连续视角旋转图像
- 重建的3D网格完整度和保真度优于当前最佳方法
- 适合需要高质量3D人物建模的应用场景
我们提出一种从单张输入图像生成人物360°环绕视频的方法。现有方法通常采用基于图像的扩散模型进行多视角合成，但各视角间结果不一致且与原始身份偏差较大。相比之下，近期视频扩散模型已展现出与提示高度对齐的逼真生成能力。受此启发，我们提出HumanOrbit——一个用于多视角人体图像生成的视频扩散模型。该方法使模型能够合成围绕主体的连续相机旋转画面，在保持人物外观与身份一致性的同时实现几何一致的新视角生成。利用生成的多视角帧，我们进一步设计了一条重建流水线，恢复出带纹理的三维网格。实验验证了HumanOrbit在多视角图像生成上的有效性，且重建的3D模型在完整性和保真度上均优于现有最先进基线方法。
原文摘要 · Abstract (English)
We present a method for generating a full 360° orbit video around a person from a single input image. Existing methods typically adapt image-based diffusion models for multi-view synthesis, but yield inconsistent results across views and with the original identity. In contrast, recent video diffusion models have demonstrated their ability in generating photorealistic results that align well with the given prompts. Inspired by these results, we propose HumanOrbit, a video diffusion model for multi-view human image generation. Our approach enables the model to synthesize continuous camera rotations around the subject, producing geometrically consistent novel views while preserving the appearance and identity of the person. Using the generated multi-view frames, we further propose a reconstruction pipeline that recovers a textured mesh of the subject. Experimental results validate the effectiveness of HumanOrbit for multi-view image generation and that the reconstructed 3D models exhibit superior completeness and fidelity compared to those from state-of-the-art baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。