让新物体在3D中更自然，用动态专家选择避免形状断裂
GUMBEL-NERF: Representing Unseen Objects as Part-Compositional Neural Radiance Fields
- 采用事后选择专家的机制，避免边界处形状不连续
- 在单张/少量输入下合成视角质量优于基线方法
- 适合需要高保真重建新物体的3D生成任务
我们提出 Gumbel-NeRF，一种基于混合专家（MoE）结构的神经辐射场模型，通过事后专家选择机制实现未见过物体的新视角合成。以往研究表明，MoE NeRF 能够高质量表示包含多个物体的大规模场景，但在处理从单张或少数输入重建未知物体时，模型在专家边界附近常出现低质量表现。我们发现，这一问题主要源于前瞻式专家选择机制，导致物体形状在边界处产生不自然的不连续。Gumbel-NeRF 改用事后选择机制，确保密度场在专家边界附近仍保持连续性。在 SRN cars 数据集上的实验表明，该方法在多种图像质量指标上均优于现有基线。
原文摘要 · Abstract (English)
We propose Gumbel-NeRF, a mixture-of-expert (MoE) neural radiance fields (NeRF) model with a hindsight expert selection mechanism for synthesizing novel views of unseen objects. Previous studies have shown that the MoE structure provides high-quality representations of a given large-scale scene consisting of many objects. However, we observe that such a MoE NeRF model often produces low-quality representations in the vicinity of experts' boundaries when applied to the task of novel view synthesis of an unseen object from one/few-shot input. We find that this deterioration is primarily caused by the foresight expert selection mechanism, which may leave an unnatural discontinuity in the object shape near the experts' boundaries. Gumbel-NeRF adopts a hindsight expert selection mechanism, which guarantees continuity in the density field even near the experts' boundaries. Experiments using the SRN cars dataset demonstrate the superiority of Gumbel-NeRF over the baselines in terms of various image quality metrics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。