边缘机器人用大模型动态学认地标，边走边记边干活。
ATLASv2: LLM-Guided Adaptive Landmark Acquisition and Navigation on the Edge
- 用微调小大模型实时识物存地标，边学边导航。
- 真实场景下指令理解与任务执行成功率高。
- 全机载运行，延迟低功耗小，适合真环境部署。
部署在边缘设备上的自主系统面临资源受限、实时性要求高及动态环境适应难题。本文提出ATLASv2，一个集成微调后的TinyLLM、实时目标检测与高效路径规划的新型系统，可在边缘设备Jetson Nano上实现分层多任务导航与操作。ATLASv2通过检测并定位环境中的物体，动态扩展可导航地标，并将其存入内部知识库以供后续任务使用。我们在自建的家庭与办公室真实场景中评估该系统，包含多样物体与地标。结果表明，ATLASv2能有效解析自然语言指令，将其分解为底层动作并成功执行任务。通过在全机载框架中引入生成式AI，ATLASv2实现了优化的资源利用，提示延迟与功耗极低，缩小了仿真环境与真实应用之间的差距。
原文摘要 · Abstract (English)
Autonomous systems deployed on edge devices face significant challenges, including resource constraints, real-time processing demands, and adapting to dynamic environments. This work introduces ATLASv2, a novel system that integrates a fine-tuned TinyLLM, real-time object detection, and efficient path planning to enable hierarchical, multi-task navigation and manipulation all on the edge device, Jetson Nano. ATLASv2 dynamically expands its navigable landmarks by detecting and localizing objects in the environment which are saved to its internal knowledge base to be used for future task execution. We evaluate ATLASv2 in real-world environments, including a handcrafted home and office setting constructed with diverse objects and landmarks. Results show that ATLASv2 effectively interprets natural language instructions, decomposes them into low-level actions, and executes tasks with high success rates. By leveraging generative AI in a fully on-board framework, ATLASv2 achieves optimized resource utilization with minimal prompting latency and power consumption, bridging the gap between simulated environments and real-world applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。