用AI代理自动完成建筑建模，提升效率并保留设计意图。
BIMgent: Towards Autonomous Building Modeling via Computer-use Agents
- 基于多模态大模型的智能代理，通过操作GUI实现自主建模。
- 在真实任务中达成32%成功率，基线模型全失败（0%）。
- 适合建筑行业自动化建模，降低人工负担，适配真实项目场景。
现有计算机使用代理主要聚焦通用桌面自动化任务，对高度专业领域的应用探索有限。建筑、工程与施工（AEC）领域的三维建筑建模涉及开放式设计任务和复杂的BIM软件交互模式，当前研究尚未充分覆盖。本文提出BIMgent，一种由多模态大语言模型驱动的智能体框架，可通过图形用户界面（GUI）操作实现自主建筑建模。BIMgent自动化完成建筑建模全流程，包括多模态概念设计输入、软件特定流程规划及高效执行建模操作。我们在真实建筑建模任务上评估了BIMgent，涵盖文本驱动的概念设计生成与既有设计重建。结果表明，其生成的设计质量合理，操作成功率达32%，而所有基线模型均未能完成任务（成功率0%）。实验显示，BIMgent有效降低人工工作量，同时保持设计意图，展现出在实际建筑建模场景中的部署潜力。
原文摘要 · Abstract (English)
Existing computer-use agents primarily focus on general-purpose desktop automation tasks, with limited exploration of their application in highly specialized domains. In particular, the 3D building modeling process in the Architecture, Engineering, and Construction (AEC) sector involves open-ended design tasks and complex interaction patterns within Building Information Modeling (BIM) authoring software, which has yet to be thoroughly addressed by current studies. In this paper, we propose BIMgent, an agentic framework powered by multimodal large language models (LLMs), designed to enable autonomous building model authoring via graphical user interface (GUI) operations. BIMgent automates the architectural building modeling process, including multimodal input for conceptual design, planning of software-specific workflows, and efficient execution of the authoring GUI actions. We evaluate BIMgent on real-world building modeling tasks, including both text-based conceptual design generation and reconstruction from existing building design. The design quality achieved by BIMgent was found to be reasonable. Its operations achieved a 32% success rate, whereas all baseline models failed to complete the tasks (0% success rate). Results demonstrate that BIMgent effectively reduces manual workload while preserving design intent, highlighting its potential for practical deployment in real-world architectural modeling scenarios. Project page: https://tumcms.github.io/BIMgent.github.io/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。