用文字生成3D场景，还能自由编辑物体位置和动态。
DreamScene: 3D Gaussian-based End-to-end Text-to-3D Scene Generation
- 用GPT-4规划物体布局，生成无碰撞的结构化场景图。
- 通过多阶段采样与优化，快速合成高保真3D物体几何。
- 支持室内室外统一渲染，可实现物体移动与4D动画编辑。
从自然语言生成3D场景在游戏、影视和设计领域具有广阔前景，但现有方法在自动化、3D一致性及精细控制方面仍存挑战。本文提出DreamScene，一个端到端的高质量、可编辑3D场景生成框架，支持文本或对话输入。系统首先由GPT-4代理推断物体语义与空间约束，构建混合图结构；随后基于图的布局算法生成无碰撞的结构化布局。在此基础上，采用生成式形态采样（FPS）结合多时间步采样与重建优化，实现快速且逼真的物体几何生成。为保障全局一致性，引入针对室内外场景定制的渐进式相机采样策略。最终系统支持细粒度编辑，包括物体移动、外观更改及4D动态变化。实验表明，DreamScene在质量、一致性和灵活性上均优于现有方法，为开放域3D内容创作提供实用解决方案。代码与演示见https://jahnsonblack.github.io/DreamScene-Full/。
原文摘要 · Abstract (English)
Generating 3D scenes from natural language holds great promise for applications in gaming, film, and design. However, existing methods struggle with automation, 3D consistency, and fine-grained control. We present DreamScene, an end-to-end framework for high-quality and editable 3D scene generation from text or dialogue. DreamScene begins with a scene planning module, where a GPT-4 agent infers object semantics and spatial constraints to construct a hybrid graph. A graph-based placement algorithm then produces a structured, collision-free layout. Based on this layout, Formation Pattern Sampling (FPS) generates object geometry using multi-timestep sampling and reconstructive optimization, enabling fast and realistic synthesis. To ensure global consistent, DreamScene employs a progressive camera sampling strategy tailored to both indoor and outdoor settings. Finally, the system supports fine-grained scene editing, including object movement, appearance changes, and 4D dynamic motion. Experiments demonstrate that DreamScene surpasses prior methods in quality, consistency, and flexibility, offering a practical solution for open-domain 3D content creation. Code and demos are available at https://jahnsonblack.github.io/DreamScene-Full/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。