无需训练即可将动态场景风格化，保持画面时空一致性。
ZDySS -- Zero-Shot Dynamic Scene Stylization using Gaussian Splatting
- 用高斯点阵表示场景，通过特征向量实现零样本风格迁移。
- 在真实动态场景上表现优于现有方法，时空一致性更强。
- 适合游戏、影视等需快速风格切换的实时应用。
基于参考图像对动态场景进行风格化，在游戏、影视及增强/虚拟现实等实际应用中至关重要。然而，实现空间与时间维度上的稳定风格化仍是重大挑战。现有多数方法针对静态场景设计，且需为每张风格图单独优化，适应性差。本文提出ZDySS，一种面向动态场景的零样本风格化框架，可在推理时泛化至未见过的风格图像。该方法采用高斯点阵表示场景，每个高斯点关联一个可学习特征向量，可生成任意视角和时间戳下的特征图。通过在学习到的特征向量上应用风格迁移，而非渲染后的特征图，提升了帧间时空一致性。在真实动态场景测试中，本方法性能和连贯性均优于当前最优基线，为实际应用提供了稳健解决方案。
原文摘要 · Abstract (English)
Stylizing a dynamic scene based on an exemplar image is critical for various real-world applications, including gaming, filmmaking, and augmented and virtual reality. However, achieving consistent stylization across both spatial and temporal dimensions remains a significant challenge. Most existing methods are designed for static scenes and often require an optimization process for each style image, limiting their adaptability. We introduce ZDySS, a zero-shot stylization framework for dynamic scenes, allowing our model to generalize to previously unseen style images at inference. Our approach employs Gaussian splatting for scene representation, linking each Gaussian to a learned feature vector that renders a feature map for any given view and timestamp. By applying style transfer on the learned feature vectors instead of the rendered feature map, we enhance spatio-temporal consistency across frames. Our method demonstrates superior performance and coherence over state-of-the-art baselines in tests on real-world dynamic scenes, making it a robust solution for practical applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。