arXiv:2606.00162cs.ROcs.CV2026-06中稿 · 2026 IEEE 22nd Int…

用构建系统思路优化机器人数据集生成,提速超380倍。

Modeling Robotics Dataset Construction as an Artifact-Based Build Process

  • 将数据集构建建模为依赖图上的构件过程,支持增量更新。
  • 在20.4GB数据上,迭代任务快386倍,整体运行时间显著降低。
  • 开源工具Bagzel支持可复现、高效的数据集生成,适合研究者和工程师。

机器人系统产生大量多模态传感器数据,但将ROS bag记录转换为机器学习数据集通常依赖临时编写的顺序脚本,导致工程负担重、迭代周期长。本文将数据集构建建模为基于构件的构建过程,依托依赖图实现可复现、增量式数据生成,并在Bagzel(一个开源Bazel扩展)中实现,支持nuScenes格式导出。对比顺序式rosbag2nuscenes基线,Bagzel在所有执行模式下均减少运行时间,尤其在迭代工作流中表现突出:20.4 GB数据集上暖启动构建加速达386.26倍,增量构建提升7.21倍。在5.1至20.4 GB的数据规模下,Bagzel系列方案表现出更优的扩展性,尤其在暖启动与增量模式。Bagzel-xattr(服务器端摘要管理)进一步提升性能,相较Bagzel平均减少5.9%运行时间。总体而言,将机器人数据集构建建模为构件化构建过程,大幅降低更新延迟,同时保障确定性与可复现性。Bagzel已开源,地址:https://github.com/UniBwTAS/bagzel。

原文摘要 · Abstract (English)

Robotic systems generate large volumes of multimodal sensor data, but converting ROS bag recordings into machine learning datasets is often handled by ad hoc sequential scripts, creating engineering overhead and slow iteration cycles. We model dataset construction as an artifact-based build process over a dependency graph and implement this approach in Bagzel, an open-source Bazel extension for reproducible, incremental dataset generation (including nuScenes-format export). We compare Bagzel and Bagzel-xattr (server-side digest management) against a sequential rosbag2nuscenes baseline. Bagzel reduces runtime in all evaluated execution modes, with the largest gains in iterative workflows (up to 386.26x in warm builds and 7.21x in incremental builds on a 20.4 GB dataset). Across dataset sizes from 5.1 to 20.4 GB, Bagzel variants show markedly better scaling behavior than the baseline, especially in warm and incremental modes. Bagzel-xattr provides additional gains, with a mean runtime reduction of 5.9% compared to Bagzel in the input granularity study. Overall, modeling robotics dataset construction as an artifact-based build process substantially reduces dataset update latency while maintaining a deterministic build design that supports reproducibility. Bagzel is publicly available at https://github.com/UniBwTAS/bagzel.

数据集构建机器人学习增量计算Bazel

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。