首个真实草莓采摘场景的6D姿态数据集,解决仿真到现实的迁移难题。
From Simulation to the Real-World: An In-Field 6D Pose Dataset and Baseline for Robotic Strawberry Harvesting

- 通过相机位姿恢复与三维重建,构建真实农田中草莓的6D姿态标注
- 合成数据训练模型在真实场景表现差,加入少量真实数据显著提升旋转精度
- 首次公开真实世界草莓6D姿态数据集,适合机器人采摘研究者使用
机器人草莓采摘需要精确的6D姿态估计,但真实农田中获取6D姿态真值极具挑战。现有研究主要依赖合成数据,缺乏真实场景性能评估。本文通过PnP恢复相机位姿,进行毫米级场景重建,并为每颗草莓标注单个3D边界框并跨帧传播,构建了目前已知首个在实际草莓农场采集的真实世界红熟期草莓6D姿态真值数据集(12,040张图像)。同时引入在NVIDIA Isaac Sim中渲染的合成数据集,具备场景级真实感与域随机化。尽管仿真环境改进,仅用合成数据训练的模型仍无法有效迁移到真实图像;加入少量真实数据后,所有骨干网络的平移与旋转精度均提升。在单目RGB设置下,引入真实数据后旋转基本可恢复,而深度估计仍是限制姿态精度的关键。该系列基线结果为后续研究提供参考。真实数据集已公开于https://huggingface.co/datasets/WoojungSon/FieldStraw6D,数据采集流程见https://github.com/wjson2435/FieldStraw6D-pipeline。
原文摘要 · Abstract (English)
Robotic strawberry harvesting requires precise 6D pose estimation; however, collecting 6D pose ground truth in real agricultural fields is inherently challenging. Existing strawberry 6D pose estimation studies have therefore relied mainly on synthetic data, leaving their in-field performance unquantified. In this work, we obtain ground truth indirectly, by recovering camera poses via PnP, reconstructing each scene at metric scale, and annotating a single 3D bounding box per strawberry that is propagated across all frames, yielding, to the best of our knowledge, the first real world 6D pose ground truth dataset of red-stage strawberries collected at an actual strawberry farm (12,040 images). We also introduce a synthetic dataset rendered in NVIDIA Isaac Sim, featuring scene-level realism and domainrandomization. Despite this improved simulation setup, models trained on synthetic data alone fail to transfer to in-field images, while introducing a small amount of real data improves both translation and rotation accuracy across all backbones. Under the monocular RGB setting evaluated here, rotation is largely recovered once real data is used, and depth is what limits pose accuracy. These baselines across backbone encoders serve as a reference for future work. The real-world dataset is publicly available at https://huggingface.co/datasets/WoojungSon/F ieldStraw6D, and the data-collection pipeline is available at https://github.com/wjson2435/FieldStraw6D-pipeline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。