解决3D场景生成中因图像外扩导致的语义漂移问题
ScenePainter: Semantically Consistent Perpetual 3D Scene Generation with Concept Relation Alignment
- 构建分层概念图，对齐场景先验与当前理解
- 动态优化图结构，生成更连贯且多样的视图序列
- 适用于长时视频合成与3D场景重建任务
持续3D场景生成旨在生成长序列且一致的3D视角序列，适用于长期视频合成与3D场景重建。现有方法采用“导航-想象”范式，依赖外扩机制扩展连续视角，但生成序列易受外扩模块累积偏差影响，产生语义漂移。为此，本文提出ScenePainter框架，通过将外扩器的场景特定先验与当前场景理解对齐，实现语义一致的3D场景生成。具体地，引入分层图结构SceneConceptGraph，构建多层次场景概念间的关系，指导外扩器生成一致的新视角，并可动态优化以提升多样性。大量实验表明，该框架有效克服语义漂移，生成更具一致性与沉浸感的3D视图序列。
原文摘要 · Abstract (English)
Perpetual 3D scene generation aims to produce long-range and coherent 3D view sequences, which is applicable for long-term video synthesis and 3D scene reconstruction. Existing methods follow a "navigate-and-imagine" fashion and rely on outpainting for successive view expansion. However, the generated view sequences suffer from semantic drift issue derived from the accumulated deviation of the outpainting module. To tackle this challenge, we propose ScenePainter, a new framework for semantically consistent 3D scene generation, which aligns the outpainter's scene-specific prior with the comprehension of the current scene. To be specific, we introduce a hierarchical graph structure dubbed SceneConceptGraph to construct relations among multi-level scene concepts, which directs the outpainter for consistent novel views and can be dynamically refined to enhance diversity. Extensive experiments demonstrate that our framework overcomes the semantic drift issue and generates more consistent and immersive 3D view sequences. Project Page: https://xiac20.github.io/ScenePainter/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。