让机器人在遮挡下也能精准选果并定位抓取点，提升采摘成功率。
HarvestPoint-ACT: Explicit Target Selection and Harvest-Point Conditioning for Robotic Fruit Harvesting under Occlusion
- 通过关键点检测显式预测每个果实的抓取位置
- 在遮挡下仍保持88%成功率，重遮挡时达75%
- 适合复杂果园环境中的机器人采摘任务
端到端模仿学习虽避免了手动设计机械臂运动，但政策仍需决定采哪颗果、何处闭合夹爪。遮挡会导致策略丢失选定果实，仅凭像素难以推断正确闭合点。本文提出HarvestPoint-ACT，将目标选择与抓取点确定显式化，并传递给策略。前端实例分割结合关键点分支，为每个可见果实预测掩码与采摘点（即夹爪闭合位置）。调度器依据遮挡程度和移动距离对候选果实排序并选出目标；每次尝试后重新检测并重排序，以应对树冠变化。选定果实以八维状态编码输入动作分块变压器，包含绝对采摘点、夹爪至该点向量、有效性标志及置信度。当目标短暂失联时，系统保留其在机器人基坐标系下的最后估计并标记为过期，若持续丢失则中止尝试。在树冠模拟环境中，成功率达88%，重遮挡下仍达75%。
原文摘要 · Abstract (English)
End-to-end imitation learning avoids hand-made robot motion for approaching and grasping, but the policy must still decide which fruit to pick and where to close the gripper. Occlusion can make the policy lose the selected fruit during harvesting, and the correct closing point is difficult to infer from pixels alone. This paper presents HarvestPoint-ACT, which makes both decisions explicit in perception and provides them to the policy. An instance segmentation front end with a keypoint branch predicts a mask and a harvest point for each visible fruit, where the harvest point specifies the location to close the gripper. A scheduler ranks detected candidates by occlusion and travel distance and selects one target. After each attempt, it redetects and reranks the candidates because the canopy may have changed. The selected fruit is encoded for an action chunking transformer as an eight-dimensional state, containing the absolute harvest point, the vector from the gripper to that point, a validity flag, and a confidence score. When the selected fruit is temporarily undetected, the system retains the last harvest point estimate in the robot base frame and marks it as stale, and aborts the attempt if the loss persists. On a canopy mock-up, HarvestPoint-ACT achieves a success rate of 88%, and of 75% under heavy occlusion.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。