用仿真数据训练立体匹配模型，实现无人机修剪时实时精准测距。
Real-Time Branch-to-Tool Distance Estimation for Autonomous UAV Pruning: Benchmarking Five DEFOM-Stereo Variants from Simulation to Jetson Deployment
- 基于仿真数据训练五种改进版DEFOM-Stereo模型，适配无人机实时测距任务。
- 最优模型DEFOM-PrunePlus在Jetson上达3.3帧/秒，测距误差仅64.26厘米。
- 模型可从仿真直接迁移到真实场景，适合无人机安全自主修剪应用。
无人机自主修剪树木是一项高安全性要求的任务：机载感知系统需实时估算切割工具与细枝之间的物理距离，以确保无人机能无碰撞地接近、对齐并操作剪枝器。本文通过在特定任务的合成数据集上训练五种基于基础模型的立体匹配器DEFOM-Stereo变体，并将其部署于NVIDIA Jetson Orin Super 16 GB设备。数据集由Unreal Engine 5构建，包含115棵树木实例，三个视角下距离2米处采集的5,520对立体图像，使用密集EXR深度图提供完整空间监督。在合成测试集上，DEFOM-Stereo ViT-S表现最佳（均方端点误差1.74像素，D1-all 5.81%，delta-1 95.90%，深度平均绝对误差23.40厘米），但其在Jetson上的推理速度仅约2.2帧/秒（每帧450毫秒），难以满足闭环控制需求。新提出的平衡型模型DEFOM-PrunePlus（约2100万参数，Jetson上约3.3帧/秒）在可部署性上取得最佳权衡（EPE 5.87像素，深度MAE 64.26厘米，delta-1 87.59%），帧率足以支持实时引导，精度保障2米作业范围内安全接近规划。轻量级模型DEFOM-PruneStereo（约6.9帧/秒）和DEFOM-PruneNano（约8.5帧/秒）虽速度快，但深度误差超过57厘米，不适用于安全执行。对真实照片的零样本推理验证了全尺寸模型能保留枝干几何结构，证实了从仿真到真实的迁移能力。结论是：DEFOM-PrunePlus为机载距离估计提供了最实用的精度-延迟平衡，而ViT-S则作为未来硬件的性能基准。
原文摘要 · Abstract (English)
Autonomous tree pruning with unmanned aerial vehicles (UAVs) is a safety-critical real-world task: the onboard perception system must estimate the metric distance from a cutting tool to thin tree branches in real time so that the UAV can approach, align, and actuate the pruner without collision. We address this problem by training five variants of DEFOM-Stereo - a recent foundation-model-based stereo matcher - on a task-specific synthetic dataset and deploying the checkpoints on an NVIDIA Jetson Orin Super 16 GB. The training corpus is built in Unreal Engine 5 with a simulated ZED Mini stereo camera capturing 5,520 stereo pairs across 115 tree instances from three viewpoints at 2m distance; dense EXR depth maps provide exact, spatially complete supervision for thin branches. On the synthetic test set, DEFOM-Stereo ViT-S achieves the best depth-domain accuracy (EPE 1.74 px, D1-all 5.81%, delta-1 95.90%, depth MAE 23.40 cm) but its Jetson inference speed of ~2.2 FPS (~450 ms per frame) remains too slow for responsive closed-loop tool control. A newly introduced balanced variant, DEFOM-PrunePlus (~21M backbone, ~3.3 FPS on Jetson), offers the best deployable accuracy-speed trade-off (EPE 5.87 px, depth MAE 64.26 cm, delta-1 87.59%): its frame rate is sufficient for real-time guidance and its depth accuracy supports safe branch approach planning at the 2m operating range. The lightweight DEFOM-PruneStereo (~6.9 FPS) and DEFOM-PruneNano (~8.5 FPS) run fast but sacrifice substantial accuracy (depth MAE > 57 cm), making estimates too unreliable for safe actuation. Zero-shot inference on real photographs confirms that full-capacity models preserve branch geometry, validating the sim-to-real transfer. We conclude that DEFOM-PrunePlus provides the most practical accuracy-latency balance for onboard distance estimation, while ViT-S serves as the reference for future hardware.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。