arXiv:2602.05362cs.CV2026-02被引 4

用自然语言生成可编辑的高保真3D城市,提升可控性与视觉一致性。

Imagine a City: CityGenAgent for Procedural 3D City Generation

  • 分层编程生成:将城市拆解为区块与建筑程序,提升可解释性。
  • 两阶段训练:监督微调保证结构正确,强化学习优化空间与视觉对齐。
  • 支持自然语言修改,适合自动驾驶与虚拟现实场景应用。

自动化生成交互式3D城市是自动驾驶、虚拟现实和具身智能等领域的重要挑战。尽管生成模型与程序化技术的进步提升了城市生成的逼真度,现有方法仍难以兼顾高保真资产创建、可控性与可操作性。本文提出CityGenAgent,一种由自然语言驱动的分层程序化3D城市生成框架。该方法将城市生成分解为两个可解释组件:区块程序(Block Program)与建筑程序(Building Program)。为确保结构正确与语义一致,采用两阶段学习策略:(1) 监督微调(SFT),训练BlockGen与BuildingGen生成满足模式约束的合法程序,包括非自交多边形与完整字段;(2) 强化学习(RL),设计空间对齐奖励(Spatial Alignment Reward)增强空间推理能力,视觉一致性奖励(Visual Consistency Reward)弥合文本描述与视觉模态间的差距。得益于程序化表示与模型泛化能力,CityGenAgent支持自然语言编辑与操控。综合评估表明,其在语义对齐、视觉质量与可控性方面均优于现有方法,为可扩展的3D城市生成奠定了坚实基础。

原文摘要 · Abstract (English)

The automated generation of interactive 3D cities is a critical challenge with broad applications in autonomous driving, virtual reality, and embodied intelligence. While recent advances in generative models and procedural techniques have improved the realism of city generation, existing methods often struggle with high-fidelity asset creation, controllability, and manipulation. In this work, we introduce CityGenAgent, a natural language-driven framework for hierarchical procedural generation of high-quality 3D cities. Our approach decomposes city generation into two interpretable components, Block Program and Building Program. To ensure structural correctness and semantic alignment, we adopt a two-stage learning strategy: (1) Supervised Fine-Tuning (SFT). We train BlockGen and BuildingGen to generate valid programs that adhere to schema constraints, including non-self-intersecting polygons and complete fields; (2) Reinforcement Learning (RL). We design Spatial Alignment Reward to enhance spatial reasoning ability and Visual Consistency Reward to bridge the gap between textual descriptions and the visual modality. Benefiting from the programs and the models' generalization, CityGenAgent supports natural language editing and manipulation. Comprehensive evaluations demonstrate superior semantic alignment, visual quality, and controllability compared to existing methods, establishing a robust foundation for scalable 3D city generation.

3D生成程序化自然语言城市建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。