用语义辐射场构建真实场景的模拟器，支持空间推理训练与评估。
Semantic Radiance Fields as Simulators for Spatial Reasoning in Real-World Scenes
- 将2D语义分割升维为3D辐射场，融合几何、外观与语义信息
- 基于真实场景重建，支持新视角合成与语义/空占查询
- 适用于机器人空间推理任务，如果园摘苹果的物理仿真
训练和评估具身智能体的空间推理能力需要既几何精确又语义可查询的多样化环境。合成模拟器虽提供真值语义但缺乏真实感；基于真实场景重建的模拟器虽视觉逼真，却默认缺少真值语义。本文提出使用语义辐射场（Semantic Radiance Fields, SRF）作为空间推理智能体的模拟器。SRF将预训练视觉模型的2D语义分割结果提升至3D辐射场，联合编码几何、外观和每类语义身份。该表示由带位姿的RGB图像重建，支持新视角合成、语义查询和自由空间查询。由此可高效生成多样真实场景，用于训练与评估空间推理模型。以果园苹果采摘任务为例，辐射场为物理引擎提供相机渲染、语义真值和占据查询。
原文摘要 · Abstract (English)
Training and evaluating spatial reasoning in embodied agents requires diverse environments that are both geometrically faithful and semantically queryable. Synthetic simulators offer ground truth semantics but sacrifice realism; simulators based on reconstructions of real-world environments have realistic appearance but lack ground truth semantics by default. We propose using Semantic Radiance Fields (SRF) as simulators for spatial reasoning agents. SRFs are a representation that unifies these requirements by lifting 2D semantic segmentations from pretrained vision models into a 3D radiance field that jointly encodes geometry, appearance, and per-class semantic identity. The resulting fields are reconstructed from posed RGB captures of real scenes and support novel-view synthesis, semantic and free-space queries within a single grounded representation. This enables the efficient generation of diverse real-world environments to train and evaluate spatial reasoning models. As an example application, we outline an SRF-driven simulator for an orchard apple-reaching task, in which the radiance field supplies camera rendering, semantic ground truth, and occupancy queries to a physics engine.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。