从单张图重建可编辑的3D场景，支持物体与灯光的精准控制。
Engine-Native Editable 3D World Reconstruction with Objects and Lighting
- 基于视觉语言模型解析物体框与参数化灯光，实现引擎原生重建。
- 物体检测综合指标达mAP 0.1141，灯光召回率高达99.8%。
- 适合游戏开发、影视制作等需精细场景编辑的领域使用。
可编辑3D场景构建需支持物体实例与灯光的查看、移动及导入标准引擎，但现有单图方法多止于房间级几何、烘焙光照或文本生成。我们提出Lumera（光感知统一引擎原生重建与组装），一个基准与参考流程，实现单图驱动的引擎原生、光感知3D场景解析。Lumera-2K由2,513个UE5项目构建，含373万组件、6300万物体实例、10.26万引擎原生参数化灯光和9.51万相机视角。在此数据上，Lumera-Box与Lumera-Light分别适配视觉语言模型解析物体边界框与参数化灯光元组（x,y,z,r,g,b,I），结合逐物体网格重建、HDR环境估计及有限代理优化循环进行组装。在清理后的盒子基准测试中，相较DetAny3D、SpatialLM、N3D-VLM与WildDet3D，Lumera-Box取得最强总体检测、几何、语义与布局性能（合并mAP 0.1141，IoU-B 0.2472，F-score 0.2762），而WildDet3D在锚点召回上仍更优。灯光方面，Lumera-Light几乎恢复所有非空场景（召回率0.998），但个体灯光定位仍弱（F1 0.209 @0.5m）；匹配灯光中位位置误差0.261米，中位ΔE2000为4.59，强度皮尔逊相关系数r=0.628。这些结果确立参数化灯光为可量化的可编辑场景目标，并揭示关系结构、灯光召回/强度及跨引擎泛化等现存瓶颈。
原文摘要 · Abstract (English)
Editable 3D scene creation requires object instances and lights that can be inspected, moved, and imported into standard engines, yet existing single-image methods largely stop at room-scale geometry, baked/global illumination, or text-driven generation. We introduce Lumera (Light-aware Unified Engine-native Reconstruction and Assembly), a benchmark and reference pipeline for engine-native, light-aware 3D scene parsing from a single image. Lumera-2K is built from 2,513 UE5 projects and provides 3.73M components, 63M object instances, 102.6K engine-native parametric lights, and 95.1K camera views. On this data, Lumera-Box and Lumera-Light adapt VLM to parse object boxes and parametric light tuples (x,y,z,r,g,b,I), which are assembled with per-object mesh reconstruction, HDR environment estimation, and a bounded agentic refinement loop. In a sanitized box benchmark against DetAny3D, SpatialLM, N3D-VLM, and WildDet3D, Lumera-Box obtains the strongest overall detection, geometry, semantic, and layout scores (merged mAP 0.1141, IoU-B 0.2472, F-score 0.2762), while WildDet3D remains stronger on anchor recall. For lights, Lumera-Light recovers almost all non-empty scenes (recall 0.998) but remains limited at individual-light localization (F1 0.209 at 0.5 m); matched lights have median position error 0.261 m, median ΔE2000 4.59, and intensity Pearson r=0.628. These results establish parametric lights as a measurable editable-scene target and expose remaining bottlenecks in relation structure, light recall/intensity, and cross-engine generalization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。