用主动视觉与零样本学习提升机器人果园感知能力。
Enhancing Agricultural Environment Perception via Active Vision and Zero-Shot Learning
- 通过动态视角规划实现环境3D重建，自动选择最优观测点。
- 零样本模型在未知作物场景下实现快速精准分割，无需微调。
- 适合农业机器人研发者和对少样本感知感兴趣的研究者。
农业关乎人类生存，正面临前所未有的挑战。亟需高效、人机协同且可持续的耕作方式。本文结合主动视觉（AV）与零样本学习（ZSL），提升机器人在果实采摘场景下的环境感知与交互能力。基于ROS 2构建的AV流程，集成下一最佳视点（NBV）规划，通过动态3D占用地图实现3D环境重建。机器人可自主规划并移动至最具信息量的观测位置，利用ZSL模型生成的语义信息持续更新3D重建结果。仿真与真实实验表明，该系统在复杂遮挡条件下显著优于传统静态预设规划方法。采用的ZSL分割模型如YOLO World + EfficientViT SAM，展现出高速度与高精度，在无需任何微调的前提下，有效处理未知农业场景中的语义信息。
原文摘要 · Abstract (English)
Agriculture, fundamental for human sustenance, faces unprecedented challenges. The need for efficient, human-cooperative, and sustainable farming methods has never been greater. The core contributions of this work involve leveraging Active Vision (AV) techniques and Zero-Shot Learning (ZSL) to improve the robot's ability to perceive and interact with agricultural environment in the context of fruit harvesting. The AV Pipeline implemented within ROS 2 integrates the Next-Best View (NBV) Planning for 3D environment reconstruction through a dynamic 3D Occupancy Map. Our system allows the robotics arm to dynamically plan and move to the most informative viewpoints and explore the environment, updating the 3D reconstruction using semantic information produced through ZSL models. Simulation and real-world experimental results demonstrate our system's effectiveness in complex visibility conditions, outperforming traditional and static predefined planning methods. ZSL segmentation models employed, such as YOLO World + EfficientViT SAM, exhibit high-speed performance and accurate segmentation, allowing flexibility when dealing with semantic information in unknown agricultural contexts without requiring any fine-tuning process.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。