无需用户标注，一键将图像纹理精准映射到3D模型局部区域。
3D PixBrush: Image-Guided Local Texture Synthesis
- 基于参考图像自动生成定位掩码与合成纹理，实现无输入编辑。
- 定位结果全局一致且局部贴合几何形状，精度高。
- 适用于多种3D网格与图像，适合内容创作与建模领域应用。
我们提出3D PixBrush，一种在3D网格上进行图像引导的局部纹理编辑方法。该方法能预测出与参考图像中物体一致的定位掩码和合成纹理。预测的定位既全局连贯又局部精确：全局上，将参考图像中的物体上下文自动定位到输入网格上；局部上，生成的掩码严格贴合参考图像的几何结构。值得注意的是，该方法无需用户输入（如涂鸦或边界框）即可实现精准定位，而是从零开始预测3D网格上的定位掩码。为此，我们改进了分数蒸馏采样技术，引入定位与参考图像双重引导，称为定位调制图像引导。我们在多种网格与图像上验证了该方法的有效性。
原文摘要 · Abstract (English)
We present 3D PixBrush, a method for performing image-driven edits of local regions on 3D meshes. 3D PixBrush predicts a localization mask and a synthesized texture that faithfully portray the object in the reference image. Our predicted localizations are both globally coherent and locally precise. Globally - our method contextualizes the object in the reference image and automatically positions it onto the input mesh. Locally - our method produces masks that conform to the geometry of the reference image. Notably, our method does not require any user input (in the form of scribbles or bounding boxes) to achieve accurate localizations. Instead, our method predicts a localization mask on the 3D mesh from scratch. To achieve this, we propose a modification to the score distillation sampling technique which incorporates both the predicted localization and the reference image, referred to as localization-modulated image guidance. We demonstrate the effectiveness of our proposed technique on a wide variety of meshes and images.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。