提出城市环境问答新任务与智能体,实现复杂城市空间的主动探索与回答。
CityEQA: A Hierarchical LLM Agent on Embodied Question Answering Benchmark in City Space
- 构建分层智能体PMA,拆解问题、维护空间认知地图、分工执行任务。
- 在1412个任务上达到人类60.7%的准确率,显著优于基线模型。
- 适合研究城市级智能体、具身认知与长程规划的学者参考。
具身问答(EQA)主要聚焦于室内环境,而城市空间所涉及的环境、动作与感知复杂性尚未得到充分研究。为此,我们提出CityEQA,一个在动态城市空间中通过主动探索回答开放词汇问题的新任务。为支持该任务,我们构建了首个基准数据集CityEQA-EC,包含1,412个由人工标注的任务,覆盖六个类别,基于真实3D城市模拟器。此外,我们提出Planner-Manager-Actor(PMA)智能体,具备长周期规划与分层任务执行能力:Planner将问题分解为子任务,Manager维持以物体为中心的认知地图进行空间推理,Specialized Actors分别负责导航、探索与信息收集。实验表明,PMA在任务中达到60.7%的人类水平准确率,显著优于现有基线。尽管表现良好,但与人类仍有差距,凸显城市场景中视觉推理能力仍需提升。本工作为城市空间智能发展铺平道路。数据与代码已开源:https://github.com/BiluYong/CityEQA.git。
原文摘要 · Abstract (English)
Embodied Question Answering (EQA) has primarily focused on indoor environments, leaving the complexities of urban settings-spanning environment, action, and perception-largely unexplored. To bridge this gap, we introduce CityEQA, a new task where an embodied agent answers open-vocabulary questions through active exploration in dynamic city spaces. To support this task, we present CityEQA-EC, the first benchmark dataset featuring 1,412 human-annotated tasks across six categories, grounded in a realistic 3D urban simulator. Moreover, we propose Planner-Manager-Actor (PMA), a novel agent tailored for CityEQA. PMA enables long-horizon planning and hierarchical task execution: the Planner breaks down the question answering into sub-tasks, the Manager maintains an object-centric cognitive map for spatial reasoning during the process control, and the specialized Actors handle navigation, exploration, and collection sub-tasks. Experiments demonstrate that PMA achieves 60.7% of human-level answering accuracy, significantly outperforming competitive baselines. While promising, the performance gap compared to humans highlights the need for enhanced visual reasoning in CityEQA. This work paves the way for future advancements in urban spatial intelligence. Dataset and code are available at https://github.com/BiluYong/CityEQA.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。