arXiv:2411.12089cs.CVcs.GR2024-11CVPR被引 7

首次实现3D物体切割时的内部纹理实时生成,让虚拟水果切开有果肉和种子。

FruitNinja: 3D Object Interior Texture Generation with Gaussian Splatting

  • 用扩散模型逐层填充切面,结合体素网格平滑实现整体纹理连贯
  • 在任意几何变形下仍保持高质量内部视图,实时渲染无额外优化
  • 适合需要真实感3D内容的视觉设计、游戏开发与虚拟仿真领域

现实世界中,物体被切开时会露出内部纹理,但现有3D生成任务对此研究不足。例如,虚拟西瓜切开应显示果肉与瓜子。由于缺乏完整内部结构数据集,且采集所有切面不现实,生成方法成为首选。然而,当前3D生成与修复方法多关注可见外观,忽略内部纹理。为此,我们提出FruitNinja,首个可生成经历几何与拓扑变化的3D物体内部纹理的方法。该方法基于3D高斯点云(3DGS),合成表面与内部纹理,支持无需额外优化的实时切割与渲染。FruitNinja利用预训练扩散模型逐步填充截面视图,并采用体素网格平滑保证纹理一致性。其提出的OpaqueAtom GS策略通过密集分布的不透明高斯点,克服3DGS局限,避免大颗粒偏倚导致的训练不稳定及颜色突变,实现细粒度纹理。实验表明,FruitNinja显著优于现有方法,在任意几何操作下的实时内部视图呈现无可比拟的视觉质量。

原文摘要 · Abstract (English)

In the real world, objects reveal internal textures when sliced or cut, yet this behavior is not well-studied in 3D generation tasks today. For example, slicing a virtual 3D watermelon should reveal flesh and seeds. Given that no available dataset captures an object's full internal structure and collecting data from all slices is impractical, generative methods become the obvious approach. However, current 3D generation and inpainting methods often focus on visible appearance and overlook internal textures. To bridge this gap, we introduce FruitNinja, the first method to generate internal textures for 3D objects undergoing geometric and topological changes. Our approach produces objects via 3D Gaussian Splatting (3DGS) with both surface and interior textures synthesized, enabling real-time slicing and rendering without additional optimization. FruitNinja leverages a pre-trained diffusion model to progressively inpaint cross-sectional views and applies voxel-grid-based smoothing to achieve cohesive textures throughout the object. Our OpaqueAtom GS strategy overcomes 3DGS limitations by employing densely distributed opaque Gaussians, avoiding biases toward larger particles that destabilize training and sharp color transitions for fine-grained textures. Experimental results show that FruitNinja substantially outperforms existing approaches, showcasing unmatched visual quality in real-time rendered internal views across arbitrary geometry manipulations.

3D生成内部纹理高斯溅射实时渲染

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。