提出可识别并修正指令错误的农业视觉语言导航方法

Agricultural robots are playing as important roles across a wide range of tasks, nevertheless, they are still mainly depending on manual operations or fixed railways for moving. The A2A benchmark and the AgriVLN method pioneeringly extended Vision-and-Language Navigation (VLN) to the agricultural domain, successfully navigating agricultural robots from starting points to target positions following natural language instructions. However, we observed that almost all the prior VLN methods adopted an ideal assumption: The given instructions themselves were correct. This assumption did not align with the realistic scenarios, because anybody might say an instruction with mistakes, which raised us a question: If an instruction had a mistake, could an agricultural VLN agent be aware of it? To answer this question, we propose the A2A-MI benchmark, in which we follow A2A as the foundation benchmark and insert three classes of instruction mistakes. We use it to evaluate several state-of-the-art agricultural VLN agents, then observe sufficient drops across all of them, such as AgriVLN decreases SR by 57% in average and increases NE by 9% in average, from which we suggest the lacking awareness on instruction mistakes. To address this problem, we propose the IMAC module analyzing the instruction and image, to reason whether the instruction has mistakes and attempt to correct them when needed. We integrate it into the AgriVLN backbone to build our IMAC-AgriVLN method, successfully saving SR from 0.10 to 0.14 and NE from 4.81 m to 4.79 m, which demonstrates the effectiveness of IMAC on strengthening the robustness against instruction mistakes. Project: https://github.com/AlexTraveling/IMAC-AgriVLN.