让3D场景按文本风格重绘，还能保持形状一致
Morpheus: Text-Driven 3D Gaussian Splat Shape and Color Stylization
- 用自回归方法控制风格强度，实现形状与颜色同步变化
- 通过深度引导注意力和控制网络，确保多帧风格一致
- 适合需要风格化3D内容的创作或数据增强场景
使用新视角合成探索现实世界很有趣，而以不同风格重构这些世界则带来更多乐趣。风格化场景还可用于训练数据有限的下游任务，以扩展模型的训练分布。目前大多数新视角合成风格化方法难以有效改变几何结构，因为几何修改需要更强的风格化力度，但过强会导致风格不稳定和不一致。本文提出一种新的自回归3D高斯点云风格化方法，包含一个新型RGBD扩散模型，可独立控制外观与形状的风格强度。为保证多帧间的一致性，采用深度引导交叉注意力、特征注入，以及基于合成帧的形变控制网络（Warp ControlNet）来指导新帧的风格化过程。我们通过大量定性结果、定量实验和用户研究验证了该方法的有效性。代码已公开。
原文摘要 · Abstract (English)
Exploring real-world spaces using novel-view synthesis is fun, and reimagining those worlds in a different style adds another layer of excitement. Stylized worlds can also be used for downstream tasks where there is limited training data and a need to expand a model's training distribution. Most current novel-view synthesis stylization techniques lack the ability to convincingly change geometry. This is because any geometry change requires increased style strength which is often capped for stylization stability and consistency. In this work, we propose a new autoregressive 3D Gaussian Splatting stylization method. As part of this method, we contribute a new RGBD diffusion model that allows for strength control over appearance and shape stylization. To ensure consistency across stylized frames, we use a combination of novel depth-guided cross attention, feature injection, and a Warp ControlNet conditioned on composite frames for guiding the stylization of new frames. We validate our method via extensive qualitative results, quantitative experiments, and a user study. Code online.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。