用智能体解决仓库复杂空间问答问题,效率高且无需大量数据训练。
Warehouse Spatial Question Answering with LLM Agent
- 构建基于大模型的智能体系统,结合工具调用实现空间推理。
- 在AI City挑战赛数据集上,物体检索、计数与距离估计准确率高。
- 适合需要高效空间理解的工业场景,如智能仓储与机器人导航。
空间理解是现有多模态大模型面临的难题。以往方法依赖大规模微调提升空间感知能力。本文提出一种数据高效的解决方案:构建具备强空间推理能力的LLM智能体系统,用于解决复杂室内仓库中的空间问答任务。该系统整合多种工具,使大模型能进行空间推理并调用API完成复杂空间问题的解答。在2025年AI City Challenge物理智能空间感知仓库数据集上的大量评估表明,该系统在物体检索、计数和距离估计等任务中表现出高精度与高效率。代码已开源:https://github.com/hsiangwei0903/SpatialAgent。
原文摘要 · Abstract (English)
Spatial understanding has been a challenging task for existing Multi-modal Large Language Models~(MLLMs). Previous methods leverage large-scale MLLM finetuning to enhance MLLM's spatial understanding ability. In this paper, we present a data-efficient approach. We propose a LLM agent system with strong and advanced spatial reasoning ability, which can be used to solve the challenging spatial question answering task in complex indoor warehouse scenarios. Our system integrates multiple tools that allow the LLM agent to conduct spatial reasoning and API tools interaction to answer the given complicated spatial question. Extensive evaluations on the 2025 AI City Challenge Physical AI Spatial Intelligence Warehouse dataset demonstrate that our system achieves high accuracy and efficiency in tasks such as object retrieval, counting, and distance estimation. The code is available at: https://github.com/hsiangwei0903/SpatialAgent
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。