arXiv:2608.16484cs.CV2026-08中稿 · IEEE APGARSS 2026

用大模型将遥感图像转为可编辑的代码化城市布局。

Remote-Sensing City Layout Extraction with MLLM

论文配图:Remote-Sensing City Layout Extraction with MLLM
图 1 · 摘自论文原文
  • 用多模态大模型分三步生成道路、地块和建筑及其关系。
  • 在CityLayout-100上达到41.1% mIoU和48.3% gIoU。
  • 输出可执行代码,支持3D渲染与平面图同步编辑。

遥感系统通常以检测框、语义掩码或矢量边界描述城市内容,这些输出虽能定位类别并支持图像平面评分,但无法构成保留对象身份、类型关系、拓扑结构和再生规则的可执行布局。本文提出Code-as-City,将单张俯视图的城市布局提取转化为受约束的代码生成任务,采用多模态大语言模型(MLLM)。首先由图像模型生成五类对齐的语义布局先验;随后通过三次有序的MLLM处理,结合图像与先验恢复道路、地表覆盖区域及关系、建筑;最终通过确定性归一化,将累积记录转换为城市图和受限布局程序。执行该程序可生成可渲染的3D城市布局及共享几何的正射语义投影。投影支持像素级与遥感掩码对比,同时保持命名对象、关系和编辑操作,实现双视图同步再生。在CityLayout-100的100个场景上,完整框架获得41.1%的平均交并比和48.3%的全局交并比,验证了视觉观测可被转化为可检查、可编辑的城市代码,并支持耦合的平面与3D输出。

原文摘要 · Abstract (English)

Remote-sensing systems usually describe urban content with detection boxes, semantic masks, or vector boundaries. Such outputs locate classes and support image-plane scoring, yet they do not by themselves constitute an executable layout that retains object identities, typed relations, topology, and regeneration rules. Code-as-City instead casts urban-layout extraction from a single top-down image as constrained code generation with a multimodal large language model (MLLM). An image model first produces an aligned five-class semantic layout prior. Three ordered MLLM passes use the image and this prior to recover roads, land-cover regions and relations, and buildings. Deterministic normalization converts the accumulated records into a city graph and a restricted layout program. Executing the program creates a renderable 3D city layout and an orthographic semantic projection over shared geometry. The projection admits pixel-level comparison with remote-sensing masks, while named objects, relations, and editing operations remain available for synchronized regeneration of both views. Evaluated on the 100 scenes of CityLayout-100, the complete framework obtains 41.1% mean intersection-over-union and 48.3% global intersection-over-union. This result provides quantitative evidence that visual observations can be translated into inspectable, editable city code with coupled planar and 3D outputs.

遥感城市建模大模型代码生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。