让NeRF风格迁移更真实,多级动态注入风格细节。
Multi-level Dynamic Style Transfer for NeRFs
- 分层级提取内容特征,捕捉场景多尺度结构。
- 动态注入风格特征,实现风格与内容精准融合。
- 支持全景风格迁移,适合3D内容创作者使用。
随着神经辐射场(NeRFs)在各类三维视觉任务中的应用持续扩展,基于NeRF的风格迁移方法层出不穷。然而,现有方法通常将风格统计信息直接融入原始NeRF流程,常导致内容保留与艺术化风格表现均不理想。本文提出多层级动态风格迁移方法(MDS-NeRF),重构了专为风格化设计的NeRF流程,并引入创新的动态风格注入模块。具体而言,我们设计了一种多层级特征适配器,从内容辐射场生成多层级特征网格表示,有效捕捉场景的多尺度空间结构;同时提出动态风格注入模块,学习提取相关风格特征并自适应地融合至内容模式中。风格化的多层级特征通过所提出的多层级级联解码器转化为最终的风格化视图。此外,我们将该3D风格迁移方法拓展至支持以3D风格参考进行全景风格迁移。大量实验表明,MDS-NeRF在3D风格迁移任务中表现优异,既保留了多尺度空间结构,又能有效传递风格特征。
原文摘要 · Abstract (English)
As the application of neural radiance fields (NeRFs) in various 3D vision tasks continues to expand, numerous NeRF-based style transfer techniques have been developed. However, existing methods typically integrate style statistics into the original NeRF pipeline, often leading to suboptimal results in both content preservation and artistic stylization. In this paper, we present multi-level dynamic style transfer for NeRFs (MDS-NeRF), a novel approach that reengineers the NeRF pipeline specifically for stylization and incorporates an innovative dynamic style injection module. Particularly, we propose a multi-level feature adaptor that helps generate a multi-level feature grid representation from the content radiance field, effectively capturing the multi-scale spatial structure of the scene. In addition, we present a dynamic style injection module that learns to extract relevant style features and adaptively integrates them into the content patterns. The stylized multi-level features are then transformed into the final stylized view through our proposed multi-level cascade decoder. Furthermore, we extend our 3D style transfer method to support omni-view style transfer using 3D style references. Extensive experiments demonstrate that MDS-NeRF achieves outstanding performance for 3D style transfer, preserving multi-scale spatial structures while effectively transferring stylistic characteristics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。