构建首个大规模多模态航天器感知数据集,助力太空自主操作
SpaceSense-Bench: A Large-Scale Multi-Modal Benchmark for Spacecraft Perception and Pose Estimation
- 基于虚幻引擎5生成136个卫星模型的高保真仿真数据
- 每帧含高清图像、毫米级深度图与256束激光点云,标注7类部件语义
- 揭示小部件识别与零样本泛化仍是当前技术难点
自主空间作业如在轨服务与主动碎片清除需要对目标航天器进行精细的部件级语义理解与精准相对导航,但受成本与访问限制,轨道实测数据难以大规模获取。现有合成数据集存在目标多样性不足、单模态感知、标注不完整等问题。我们提出「SpaceSense-Bench」,一个大规模多模态航天器感知基准,包含136个卫星模型,约70GB数据。每帧提供时间同步的1024×1024 RGB图像、毫米级精度深度图及256束激光点云,并附带像素级与点级的7类部件语义标签以及精确的6-DoF姿态真值。数据通过虚幻引擎5构建的高保真空间模拟环境,配合全自动数据采集、多阶段质量控制与主流格式转换流程生成。我们对五项代表性任务(目标检测、2D语义分割、RGB-LiDAR融合3D点云分割、单目深度估计、姿态估计)进行了基准测试,发现:(i) 小尺度部件(如推进器、全向天线)感知与零样本泛化仍是当前方法的关键瓶颈;(ii) 增加训练卫星数量可显著提升对新目标的性能,凸显大规模多样数据对空间感知研究的价值。数据集、代码与工具包已公开于 https://github.com/wuaodi/SpaceSense-Bench。
原文摘要 · Abstract (English)
Autonomous space operations such as on-orbit servicing and active debris removal demand robust part-level semantic understanding and precise relative navigation of target spacecraft, yet collecting large-scale real data in orbit remains impractical due to cost and access constraints. Existing synthetic datasets, moreover, suffer from limited target diversity, single-modality sensing, and incomplete ground-truth annotations. We present \textbf{SpaceSense-Bench}, a large-scale multi-modal benchmark for spacecraft perception encompassing 136~satellite models with approximately 70~GB of data. Each frame provides time-synchronized 1024$\times$1024 RGB images, millimeter-precision depth maps, and 256-beam LiDAR point clouds, together with dense 7-class part-level semantic labels at both the pixel and point level as well as accurate 6-DoF pose ground truth. The dataset is generated through a high-fidelity space simulation built in Unreal Engine~5 and a fully automated pipeline covering data acquisition, multi-stage quality control, and conversion to mainstream formats. We benchmark five representative tasks (object detection, 2D semantic segmentation, RGB--LiDAR fusion-based 3D point cloud segmentation, monocular depth estimation, and orientation estimation) and identify two key findings: (i)~perceiving small-scale components (\emph{e.g.}, thrusters and omni-antennas) and generalizing to entirely unseen spacecraft in a zero-shot setting remain critical bottlenecks for current methods, and (ii)~scaling up the number of training satellites yields substantial performance gains on novel targets, underscoring the value of large-scale, diverse datasets for space perception research. The dataset, code, and toolkit are publicly available at https://github.com/wuaodi/SpaceSense-Bench.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。