用高斯点云渲染生成逼真合成图像，仅需一段视频即可自动训练。
Cut-and-Splat: Leveraging Gaussian Splatting for Synthetic Data Generation
- 通过高斯点云重建目标物体，自动从视频中提取实例
- 在随机背景上渲染物体，结合单目深度实现自然姿态摆放
- 生成数据真实感强，适合训练实例分割模型
合成图像生成是低成本获取标注数据的有效方法，但准确的3D建模和真实光照、相机伪影模拟仍是难点。本文提出利用高斯点云渲染（Gaussian Splatting）技术构建合成数据流水线，仅需目标物体的视频即可全自动生成高质量上下文感知的实例分割训练数据。先训练目标物体的高斯点云模型并自动从视频中提取对象，再将其渲染至随机背景，并结合单目深度估计实现自然姿态摆放。我们构建了一个新数据集验证该方法，在真实感和任务性能上优于剪贴（Cut-and-Paste）和基于扩散模型的生成方法。
原文摘要 · Abstract (English)
Generating synthetic images is a useful method for cheaply obtaining labeled data for training computer vision models. However, obtaining accurate 3D models of relevant objects is necessary, and the resulting images often have a gap in realism due to challenges in simulating lighting effects and camera artifacts. We propose using the novel view synthesis method called Gaussian Splatting to address these challenges. We have developed a synthetic data pipeline for generating high-quality context-aware instance segmentation training data for specific objects. This process is fully automated, requiring only a video of the target object. We train a Gaussian Splatting model of the target object and automatically extract the object from the video. Leveraging Gaussian Splatting, we then render the object on a random background image, and monocular depth estimation is employed to place the object in a believable pose. We introduce a novel dataset to validate our approach and show superior performance over other data generation approaches, such as Cut-and-Paste and Diffusion model-based generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。